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Report of the Auditor General 

to the House of Commons 

for April 1999 


Foreword 


I am pleased to present the April volume of my 1999 Report. Bound with this Foreword are the Main Points of 
10 separately issued chapters: 


le Correctional Service Canada — Reintegration of Offenders 

Z Revenue Canada — Underground Economy Initiative 

5s Statistics Canada — Managing the Quality of Statistics 

4. Fisheries and Oceans — Managing Atlantic Shellfish in a Sustainable Manner 
Bye Collaborative Arrangements: Issues for the Federal Government 


6. Human Resources Development Canada — Accountability for Shared Social Programs: National Child 
Benefit and Employability Assistance for People with Disabilities 


Gf. The Atlantic Groundfish Strategy: Contributions and Grants 

8. The Atlantic Groundfish Strategy: Follow-up 

2): Management of Science and Technology Personnel: Follow-up 

10. Indian and Northern Affairs Canada — Funding Arrangements for First Nations: Follow—up 


The Main Points are followed by an audit observation on the roles of National Defence and Health Canada in the 
pre-licensing use of an anti-malarial drug. (Audit observations are specific matters that come to our attention 
during our financial and compliance audits of the Public Accounts of Canada and that we believe should be 
brought to Parliament’s attention.) 


In last December’s “Matters of Special Importance — 1998” I noted that in moving from a traditional bureaucratic 
model to more flexible forms of management, the government needed to keep its focus on serving the public 
interest, achieving objectives, ensuring accountability and maintaining transparency. Directly or indirectly, the 
chapters in this volume touch on all of these issues. 


In particular, we note that they are basic elements of collaborative arrangements. Given the government's 
increasing use of these arrangements to deliver services and programs, we have set out a framework for assessing 
them. We looked at two such arrangements that are still in their early stages, and have suggested some 
improvements that can be made now to head off potential problems in the future. 


We also report on several issues that we have covered in previous reports. In some cases, the government has 
taken action to improve its performance and has made good progress. In other areas it still has some distance to 
go. The ability to build on lessons learned from past experience and from other jurisdictions emerges throughout 
the Report as a key factor in getting the most for the taxpayer’s dollar. 
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Correctional Service Canada 


Reintegration of Offenders 


Chapter 1 - Main Points 


1.1 Correctional Service Canada has made a concerted effort to respond to our 1994 and 1996 observations 
concerning the management of its offender reintegration activities. However, there are some important areas that 
require further improvement. 


12 The Service is now moving in the right direction. It has recently implemented change initiatives in 
several areas. Among them, it has strengthened the ability of national headquarters to direct and co-ordinate 
offender reintegration activities Service-wide; implemented a major initiative to streamline its reintegration 
operations; achieved international recognition for some of its offender rehabilitation programs; and improved its 
ability to measure the results and performance of its reintegration activities. 


LS Progress notwithstanding, improvement is still needed in some key areas: 
e more timely acquisition of official documents for initial offender assessment; 
e more timely casework preparation to meet the offender’s first parole date; 
e clear operational strategy for offender employment programs; 
e __ better-quality offender reintegration reports for the National Parole Board; and 


e improved adherence to national standards for frequency of contact with offenders in the community. 


Background and other observations 


1.4 Correctional Service has as one of its main responsibilities the safe reintegration of offenders into the 
community. This entails assessing offender risk and needs; preparing the offender for release into the community; 
reassessing offender suitability for release and making a recommendation to the National Parole Board; and 
providing supervision and programs for offenders in the community until the end of the sentence. 


is) Overall spending on reintegration has risen by $38 million (13 percent) over the past three fiscal years. 
Correctional Service Canada spends about $329 million or 28 percent of its total expenditures for the reintegration 
of offenders. 


1.6 In 1997-98, there were 13,449 incarcerated offenders in federal institutions and 8,744 offenders in the 
community, most of whom were supervised by Correctional Service parole officers. Until recently, the proportions 
of federal offenders in institutions and under community supervision have remained fairly constant. However, in 
1997-98, the number of offenders supervised in the community increased by about 500. 


1 By Under the Corrections and Conditional Release Act, there are several different ways that an offender can 
be released into the community: day parole (six months prior to full parole); full parole (at one third of the 
sentence) and statutory release (after two thirds of the sentence). Some offenders will be detained until the end of 
their sentence. 


1.8 A recent change allows offenders serving their first federal sentence who have not been convicted of a 
violent crime or serious drug offence to be released on day parole at one sixth of their sentence (accelerated parole 
review). 
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| ie Our previous audit work, in 1994 and 1996, identified systemic weaknesses in the Service’s management 
of reintegration activities. Those audits identified concerns in such areas as work standards, quality assurance 
procedures, performance information, implementing basic changes and learning from successes and failures. 


This chapter revisits those issues. 


1.10 As promised to the Public Accounts Committee in April 1998, we reviewed changes made by the 
Service to the custody rating scale and looked at whether the Service had implemented the new offender security 
reclassification instrument. We found that the changes to the custody rating scale reduced overrides with minimal 
impact on the number of escapes. The Service has just implemented a new reclassification instrument, as 


promised. 


1.11 In addition to their institutional security responsibilities, senior correctional officers still do not 
consistently perform their required offender reintegration duties, a necessary input to offender assessment reports 
to the National Parole Board. The Service has undertaken an initiative to address this issue. 


1.12 While the Service has developed a continuum of rehabilitation programs from the institution to the 
community, its ability to deliver these programs to offenders in the community falls short of current needs. 
Research indicates that many intervention programs that deal with offenders’ criminogenic needs are more 
effective when delivered in the community. 


Correctional Service’s responses to our recommendations are included in this chapter. The Service concurs 
with the recommendations made and its responses indicate its commitment to take the necessary corrective 
action. 
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Revenue Canada 


Underground Economy Initiative 


Chapter 2 - Main Points 


2.1 Revenue Canada has reported that the tax impact (taxes resulting from enforcement actions) of its 
activities to address the underground economy was $2.5 billion over five years; however, this includes the results 
of both regular ongoing enforcement programs and the Underground Economy Initiative. The actual tax impact 
attributable to the detection of unreported income by the 1,000 staff allocated to Initiative audit activities is much 
less than the $500 million reported. 


Ded It is difficult to assess the overall success of the Initiative in combatting the underground economy 
because the Department does not measure and report on the full range of Initiative activities and how they have 
changed taxpayer behaviour. 


2.3 Revenue Canada needs to assess the role that social marketing might play in making the public aware of 
the societal costs of unpaid taxes and in soliciting its support to combat the underground economy. Polls indicate 
that an alarming number of Canadians would be willing to participate in the underground economy. The 
Department also needs to strengthen the activities that promote voluntary compliance by businesses. 


2.4 The underground economy is a difficult and complex problem to solve and the size of the tax loss is 
significant. The problem requires continuous attention and sustained efforts from Revenue Canada and all 
Canadians. 


Background and other observations 


25 The underground economy results in tax evasion and represents an estimated annual loss of federal and 
provincial tax revenues of $12 billion. Tax evasion is not a victimless crime. It puts honest businesses at a 
competitive disadvantage and, in some cases, out of business. It also causes honest taxpayers to bear the tax load 
of those who cheat. If left unchecked, underground economy activity could lead to a loss of faith in the fairness of 
Canada’s tax system. 


2.6 In 1993, Revenue Canada announced a new initiative to combat the underground economy by allocating 
200 staff to its non-filers and non-registrants program and 1,000 staff to the audit of small businesses, where most 
of the underground economy activity exists. Thirty-five percent of the Department’s audit staff for small and 
medium-sized businesses are now involved in the Underground Economy Initiative audit activities. 


2.7 The Initiative as planned was, in our view, a balanced approach to combatting tax evasion in the 
underground economy. It included activities to promote voluntary compliance in small businesses such as 
community visits and consultations with industry associations. As well, it involved other federal departments, 
provincial and municipal governments and private sector organizations in sharing information to better deal with 
the underground economy. 


2.8 The Department can improve its targeting of audits for the detection and reassessment of unreported 
income. As well, legislative opportunities exist to strengthen existing incentives to deter participation in the 
underground economy. 


Revenue Canada has agreed to take action to address our recommendations. 
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Statistics Canada 


Managing the Quality of Statistics 


Chapter 3 - Main Points 


Oc Statistics Canada is committed to producing statistics of high quality. It has put in place a wide range of 
systems and practices to build quality into its statistical programs and to maintain an environment that encourages 
a concern for quality throughout the organization. However, the quality of the statistics it produces needs to be 
better documented and reported both within and outside the Agency. The Agency needs to integrate its many 
quality-related systems and practices better and adopt a more disciplined approach to documentation. 


3.2 The Agency has employed a number of formal quality assessment mechanisms, but individual programs 
have not applied them consistently. We concluded that the mechanisms currently used do not, either individually 

or collectively, provide systematic, transparent information on the adequacy of quality management systems and 

practices in the Agency’s statistical programs or on the quality that they actually achieve. 


Background and other observations 


a Statistics Canada is responsible for collecting, compiling, analyzing and publishing statistical information 
on the economic, social and general conditions of Canada and Canadians. The statistics that the Agency produces 
support the development, implementation and evaluation of policies, programs and decision making in all sectors. 
They help us make informed decisions about such matters as where to live, what careers to pursue and how to 
vote. The Agency is widely respected among its peers, and has an international reputation for independence, 
innovation and quality that is second to none. 


3.4 Rapid social, economic and other changes have heightened the demand for reliable, objective statistical 
information on a wide variety of issues. As the demand for and use of statistics grow, their quality becomes 
increasingly important. Statistics Canada has identified six characteristics that its systems and practices for 
managing quality need to address: relevance, accuracy, timeliness, accessibility, interpretability and coherence. 


oa The Agency carried out self-assessments of four major surveys for our audit. Each assessment reached a 
positive conclusion about the overall adequacy of quality management. The work was well planned and executed. 
We concluded that in three of the four surveys, the self-assessments provide reasonable assurance that quality 
management systems and practices are adequate. In the remaining case, however, we concluded that the 


weaknesses identified and the recommendations made are more important than the self-assessment suggests, and 
deserve the attention of senior management. 


3.6 While its policy on informing users of data quality and methodology is clear and well structured, the 
Agency’s implementation of the policy is inconsistent. Consequently, users are not always appropriately informed 
of the strengths and limitations of statistics. We also noted that while the quality of statistics figures prominently 
in its commitments to Parliament for results, the Agency’s most recent Performance Report, tabled in October 
1998, provides only limited information on the quality of the statistics that it produced. 


Statistics Canada’s responses to our recommendations are included in this chapter. The Agency either agrees 
to take action or notes that initiatives are under way in the case of six of the eight recommendations we 


make. In the remaining two cases, the Agency agrees with the intent of our recommendations and indicates 
that it will consider the issues further. 
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Fisheries and Oceans 


Managing Atlantic Shellfish in a 
Sustainable Manner 


Chapter 4 - Main Points 


4.1 In October 1997, we reported on problems associated with the Department’s management of the Atlantic 
groundfish fisheries. In the current audit, we found that many of these problems also exist in the Department’s 
management of the Atlantic shellfish fisheries. For example, we noted increases in harvesting capacity and the 
encouragement of increased fisher participation through open access licensing in the shellfish fisheries. In 
addition, we found weaknesses in the information used in making resource decisions, and gaps in monitoring, 
control and surveillance. The full impact of these problems is not obvious, as most shellfish fisheries are currently 
recording high landed values. However, in our view these are significant concerns that must be addressed to 
ensure that the shellfish fisheries are managed in a sustainable manner. 


4.2 The Department’s decisions have a profound impact on those engaged in the fishing industry and the 
communities that rely on the income generated from the industry. The absence of a fisheries policy that fully 
reflects sustainability concepts means that decisions are made on an ad hoc and inconsistent basis rather than as 
part of an overall framework for achieving a sustainable fishery. An open and transparent process in which clearly 
articulated and consistently applied principles guide decision making would provide all stakeholders with 
assurance that their interests are considered and that the resource is protected over the long term. 


4.3 We observed resource use decisions that are not consistent with the Department’s currently stated 
objectives for fisheries management. As we reported in October 1997, there is a need to have the government 
clarify fisheries objectives in legislation. The Department needs to move forward with the development of a 
sustainable fisheries framework that incorporates the interdependent factors — biological, economic and social — 
that affect the fishery. 


Background and other observations 


4.4 In the 1990s, Atlantic Canada saw a virtual collapse of its commercial groundfish fishery (cod, haddock, 
pollock, halibut and various flatfish). In the same period, however, there was a general rise in the value of 
shellfish landings (lobster, scallop, snow crab and shrimp). In 1997, the landed value of all shellfish in Atlantic 
Canada was $920 million, which represented 81 percent of the landed value of all fish landed in the region. 


4.5 The Department has stated in its key parliamentary accountability documents that its objective is 
conservation, or protecting the productive capacity of the natural resource that supports the fishery. It has also 
reported that it has an economic objective, but the expected results for this objective are not stated. The 
Department has indicated that it is not responsible or accountable for social outcomes. We found that most 
resource use decisions in the shellfish fisheries are heavily influenced by social and economic factors. 


4.6 The Department’s “Fishery of the Future” strategy reflects objectives that include ensuring economically 
viable and self-reliant fisheries, over time. However, these objectives are not fully reflected in the Department's 
reporting to Parliament. We found resource use decisions in the shellfish fisheries that are inconsistent with the 
concept of an economically viable industry. 


4.7 Co-management, designed to increase industry’s role, responsibility and accountability in fisheries 
management, is an important aspect of the Department’s Fishery of the Future strategy. Participants assuming 
greater responsibility for their industry is an important element of sustainability. However, very little power 
sharing has actually occurred. In our opinion, there are weaknesses in the Department’s current approach to 
co-management. 
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4.8 The Department has recognized that there are weaknesses in the fisheries management framework in the 
existing Fisheries Act. However, amendments to deal with these weaknesses have not yet been re-introduced in 


the House of Commons. 


Fisheries and Oceans’ responses to our recommendations are included in this chapter. The Department 
either agrees to take action or notes that initiatives are under way to address three of our four 
recommendations. The Department has not indicated an intention to take action at this time on our 
recommendation that addresses co-management. 
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Collaborative Arrangements 


Issues for the Federal Government 


Chapter 5 - Main Points 


=| Collaborative arrangements are an alternative way — a potentially more innovative, cost-effective and 
efficient way — to deliver programs and services that traditionally have been provided by federal government 
departments and Crown corporations. In collaborative arrangements, the federal government, other levels of 
government and organizations in the private and voluntary sectors agree to share power and authority in decisions 
on program and service delivery. 


5.2 With the growing use of these arrangements, more taxpayer dollars are being spent and the risks need 
correspondingly more attention. The risks include arrangements set up poorly among the partners, limiting their 
chances for success; partners not meeting commitments; insufficient attention to protecting the public interest; 
insufficient transparency; and inadequate accountability. 


i We believe that serving the public interest, effective accountability and greater transparency are basic 
elements of a framework for these arrangements, and we suggest questions that parliamentarians might wish to 
raise when assessing them. 


Background and other observations 


5.4 In the desire for greater efficiency, it is very important that the federal government and its partners not 
lose sight of the public purpose behind the collaborative arrangement, and of the need to provide transparent, fair 
and equitable service to the public. 


55 Effective accountability is more complex in a collaborative arrangement. The federal government is 
accountable to Parliament for the use of federal funds and authorities, to its partners for keeping its commitments, 
and, with its partners, to the public for the results the arrangement produces. In our view, this shared 
accountability means that more parties are accountable and it in no way lessens the federal government’s 
accountability for its own responsibilities in the arrangement. 


5.6 Delivering programs and services to the public through collaborative arrangements often requires more 
transparency than traditional delivery by a government department. Because partnerships are involved, it may be 
more difficult for citizens to know who is responsible. Consequently, the federal government needs to be as open 
as possible with information about agreements, decisions and results of the arrangements. 


Bad, The Treasury Board Secretariat has informed us that it intends to continue to provide advice and to 
develop guidance on collaborative arrangements for federal departments and agencies that will address many of 
the issues identified in this study. 


eS) 
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Human Resources Development 
Canada — Accountability for 
Shared Social Programs 


National Child Benefit and 
Employability Assistance for People 
with Disabilities 


Chapter 6 — Main Points 


National Child Benefit 


6.1 The National Child Benefit (NCB) represents a new form of collaborative arrangement between 
provinces (except Quebec) and territories and the federal government. A key challenge for all involved is to assure 
taxpayers that moneys are spent for the purposes intended, with due regard to economy and efficiency and with 
appropriate means to measure and report on effectiveness. Because there are many governments involved, it is 
also necessary to respect the jurisdictional competence of the different parties. As the NCB is implemented it is 
critical that, at a minimum, there be no less accountability because it is shared than if only one jurisdiction were 
involved. 


Background and other observations 


6.2 The goal of the National Child Benefit is to reduce the depth of poverty among families with children and 
to increase parental attachment to the work force. The NCB involves no new law, contract, or contribution 
agreement, but rather an increased federal child tax benefit (the NCB supplement) for low-income families — 
about $850 million in 1998, rising to $1.7 billion by 2000. In turn, jurisdictions providing social assistance 
benefits to families may reduce their payments by the amount of the increased tax benefit. They have agreed to 


reinvest these savings in programs that have mutually agreed-upon objectives and that benefit poor families with 
children. 


6.3 One of the clear achievements of the negotiations leading to the NCB was the partners’ agreement on the 
overall goals of both the federal and the provincial elements of the program. They also have committed to a new 
kind of joint accountability to the public. What is distinctive about this commitment is that no level of government 
is more responsible for reporting on the results than another. Each is accountable for the overall program. 


6.4 It will be a significant challenge in the first few years, before evaluation results are available, to 
demonstrate precisely how the National Child Benefit has contributed to reduced depth of poverty and increased 
employment among its recipients. In assessing progress, the NCB accountability report will also need to be clear 


about any trade-offs among goals and to discuss the implications. Otherwise, readers may expect that all goals can 
be achieved at the same time. 


6.5 There are potential gaps in the quality of financial and other information. The credibility of the 
accountability reports depends on the comparability, accuracy and verifiability of information from all parties — 
not only those who negotiated the arrangement (provinces, territories and the federal government) but also those 
who are responsible for designing and implementing only specific sub-programs (Ontario municipalities and First 
Nations). This is also a challenge for audit offices that serve the partners in this arrangement. 


6.6 . We think the key ingredients in reporting the NCB’s results to the public are transparency and adequacy 
of the information. This means: 
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e appropriate care is taken to ensure that the information is credible and, at least for financial 
information, subject to audit; 


e data are adequate to determine if the program’s overall goals are being achieved; and 
e data are sufficiently comparable that outcomes of different provincial approaches can be compared. 


It must also be clear whose responsibility it is to do all this, and who will provide assurance that it has been done. 
Moreover, those charged with the responsibility must have the capacity to undertake the task. 


The Department has said that this chapter will be of considerable assistance in its work to build an 
accountability regime for the NCB. Some issues are already being addressed. The remaining issues identified 
in this study will be raised by the federal government in its capacity as co-chair of the federal-provincial- 
territorial group working on the NCB. 


Employability Assistance for People with Disabilities 


6.7 Employability Assistance for People with Disabilities (EAPD) is the other program to emerge from 
federal-provincial discussions in 1997-98 on the social union. The aim of this 50-50 cost-shared arrangement is to 
help people with disabilities overcome the barriers they face in the labour force. EAPD supports provincial and 
territorial programs and services ranging from programs that assist in the first steps toward developing skills to 
those that support a person at work so the person can keep working. Our interest in the program at this early 
juncture is that steps be taken to ensure that the information each partner will provide on its own expenditures and 
programs is credible, and permits comparisons of different approaches and assessments of the overall program’s 
effectiveness. 


Background and other observations 


6.8 Each province has signed an agreement with the federal government. Taken together, the agreements 
illustrate several of the elements of a collaborative arrangement discussed in Chapter 5 of this Report. For 
instance, the governments agree to follow a co-ordinated, participative planning process and to evaluate program 
results. 


6.9 Human Resources Development Canada (HRDC) and provincial partners have set about to solve 
implementation issues in a collaborative fashion. For example, the provinces have committed to annual 
accountability to HRDC. However, so far there is no specific commitment to the format, substance, or timing of 
any overall annual report, nor are there specific goals or targets over any time frame. We recommend that HRDC 
prepare a brief overall annual report for EAPD, comparing activities, expenditures, program outputs and 
performance of the partners. 


6.10 The partners still have important work to do. When so many jurisdictions are involved, those with 
oversight and audit responsibilities face the challenge of helping to ensure that the quality of financial and 
performance reporting is maintained and enhanced. 


The Department has said that the case study will be useful in its work with provinces. It agrees in principle 
with the recommendation to produce an overall annual report on EAPD expenditures and performance. It 
pointed out that it will be important to note that EAPD is only one part of overall programming for people 
with disabilities. 
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The Atlantic Groundfish Strategy 


Contributions and Grants 


Chapter 7 — Main Points 


eeu | We have little assurance that all contributions under The Atlantic Groundfish Strategy (TAGS) were used 
for their intended purposes. These were part of TAGS active labour adjustment-measures managed by Human 
Resources Development Canada (HRDC). 


7.2 Most of the deficiencies noted in the audit relate to a lack of diligence in assessing project proposals and 
signing contribution agreements, as well as lack of monitoring by the Department. 


Led Many of the files contained no project proposals; in others, proposals were not sufficiently developed to 
allow proper assessment. Some agreements lacked complete information, included ineligible costs, or did not 
correspond to the measure under which the project was funded. There was little evidence of on-site monitoring 
visits to examine expense records. 


Background and other observations 


7.4 The Atlantic Groundfish Strategy, in effect from 16 May 1994 to 29 August 1998, comprised measures to 
assist those affected by the groundfish moratorium. About $150 million of the $1.9 billion allocated to TAGS was 
spent on active labour adjustment measures such as training, mobility assistance, wage subsidies and employment 
bonuses. 


Ts Those expenditures were not covered in our audit of The Atlantic Groundfish Strategy, reported in 
Chapter 16 of our October 1997 Report. Following the publication of that Report, the House of Commons 
Standing Committee on Fisheries and Oceans requested that we audit the expenditures. 


7.6 Our Office and the Internal Audit Bureau (IAB) of Human Resources Development Canada agreed that 
[AB would audit the grants and contributions made under TAGS. We closely monitored and reviewed that audit so 
we could rely on the findings for this chapter. 


tea TAGS had increased the caseload of officers by some 40,000 participants over a short period of time. 
This created pressure to identify, approve and contract for an unprecedented number of projects. 


7.8 Several contributions went to projects that were not clearly related to TAGS. More than half of the 
agreements were signed after projects had begun. Expenditures were reimbursed without supporting 
documentation for the claims. For more than half of the projects reviewed, there was no evidence that the files had 
been closed, although most of those projects had been completed more than two years earlier. 


Overall, Human Resources Development Canada agrees with the findings of the audit and recognizes that a 
number of important points and concerns are raised in the chapter. Actions undertaken to address the issues 
include the development of new policies and procedures, training for managers and staff and, in new 
initiatives, securing resources to ensure sufficient monitoring. 
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The Atlantic Groundfish Strategy 


Follow-up 


Chapter 8 — Main Points 


8.1 We believe the government’s efforts to implement the recommendations in our October 1997 Report 
Chapter 16 on The Atlantic Groundfish Strategy (TAGS) have been satisfactory. 


8.2 New fishery restructuring and adjustment measures are being implemented. In contrast to our 
observations of 1997, the eligibility criteria for the new measures are clear, logical and applicable. The 
accountability framework established for the measures corrects the shortcomings we had identified in TAGS: it 
clearly defines the responsibilities of the organizations involved, sets out an overall strategic plan and provides for 
a formal co-ordination mechanism. 


Background and other observations 


8.3 The objective of TAGS (1994-98) was to restructure the fishery industry in Atlantic Canada to make it 
economically viable and environmentally sustainable. 


8.4 Close to $1.9 billion was allocated to TAGS. Most of the funding was used to provide income support to 
the some 40,000 fishers and plant workers affected by the groundfish moratorium. 


8.5 In October 1997, we urged the government to carefully examine the impact of the decisions made under 
TAGS in order to benefit from the valuable lessons that could be learned. 


8.6 This follow-up examined progress to date in addressing our 1997 recommendations. Some of our 
observations in this chapter relate to TAGS and others to the Atlantic fishery restructuring and adjustment 
measures announced in June 1998. 


8.7 Human Resources Development Canada completed an evaluation of the labour adjustment component of 
TAGS. It also reviewed the duration of eligibility of TAGS participants. The departments involved in 
implementing the new measures now have considerable information on the profile of the targeted populations, in 
sharp contrast to the situation that prevailed when TAGS was developed. 


8.8 We believe the orderly way in which the departments are implementing the new fishery restructuring and 
adjustment measures represents satisfactory progress. 
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Management of Science and 
Technology Personnel 


Follow-up 


Chapter 9 — Main Points 


on We are satisfied with the efforts made by the science and technology community to follow up on our 
audit recommendations of 1994 and the concerns we raised in our 1996 follow-up report. In our view, the 
community is showing leadership and perseverance in dealing with the human resource management issues we 
raised. 


9.2 As aresult of all the work done since 1994, the science and technology community is now in a position to 
act. But the community is faced with considerable challenges. It must give priority to resolving the oncoming 
changes in its demographic profile that could weaken or compromise the government’s science and technology 
capacity. It must tackle the dual challenges of attracting and recruiting promising young scientists and 
technologists while retaining high-calibre employees to mentor and develop the new recruits. The community is 
thus seeking new measures, tools and resources for external recruitment, as it estimates that over the next five 
years it may have to recruit between 2,500 and 3,300 employees to build a renewed and rejuvenated science and 
technology work force. 


Background and other observations 


a5 At 31 March 1998, the federal government had close to 20,000 scientific and technical employees 
working in science-based departments, agencies, Crown corporations and research establishments in the fields of 
natural sciences and engineering (hereinafter referred to as the “science and technology community”). 


9.4 The science and technology community makes an important contribution to the government’s 1996 
science and technology strategy, which focusses on sustainable job creation and economic growth, improved 
quality of life and the advancement of knowledge. 


9.5 Expenditure and work force reduction in the public service has changed the profile of the science and 
technology community and worsened the long-identified problems of rejuvenation and recruitment. The change in 
the age profile is a major challenge to the future of the community. Not only have most senior and experienced 
scientists and technologists left the government since 1994, but the youngest and most promising as well. 


9.6 Following our 1994 audit of federal science and technology activities, the community mobilized to 
develop a management framework and a results-oriented plan for human resources management in science and 
technology. It addressed such issues as the need for a more strategic approach to the management of scientific 
personnel; for more systematic renewal of scientific personnel; and for more effort to maintain the skills and 
knowledge base in research establishments. Working groups were created to study important human resource 
issues. Among their recommendations to the Science and Technology Senior Steering Committee on Human 
Resources was that new mechanisms be adopted and human resource strategies developed to improve the 
management of science and technology personnel in science-based departments and agencies. 
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+t In 1994, we pointed out the need to develop a stronger and more effective management capability. Since 
then, the community has developed a competency profile for science and technology managers. During our 
consultations, we noted that most science-based departments used their own competency profiles instead of the 
one developed by the working group. Moreover, their profiles were being used solely to identify training needs 
and generally not for purposes of manager recruitment, promotion or performance assessment. Present practices 
suggest a lack of consensus in the community on the management competency profile defined by the working 
group. This could eventually prevent the integration of recruitment and training activities as well as the reward, 
promotion and compensation systems envisioned in the Science and Technology Blueprint for Human Resources 
Management. 


9.8 The Treasury Board Secretariat, science-based departments and agencies and the science and technology 
community have indicated that they are committed to following through on the strategies and plans developed to 
date. 
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Indian and Northern Affairs 
Canada 


Funding Arrangements for First 
Nations: Follow-up 


Chapter 10 - Main Points 


10.1 Indian and Northern Affairs Canada still needs to better match the level of flexibility attached to funding 
arrangements with the willingness and ability of First Nations to assume responsibility for billions of dollars they 
receive annually through the Department. 


10.2 The Department is not taking adequate steps to ensure that allegations of wrongdoing, including 
complaints and disputes related to funding arrangements, are appropriately resolved. Redress (resolution 
mechanisms) needs to be improved as an element of accountability. 


10.3 The Department has stated that it has slowed the rate at which the Financial Transfer Arrangement (FTA), 
a new type of funding arrangement, is being implemented in order to address issues concerning the willingness 
and ability of First Nations to adopt it. The Department still has a long way to go if it wishes to achieve its 
objective of implementing the FTA as the appropriate funding mechanism to replace other types of funding 
arrangements. It will need to find ways to expedite the conversion process while improving co-ordination of 
funding with other federal departments. 


Background and other observations 


10.4 Funding arrangements are a key element in the relationship between First Nations and the federal 
government. Parliament appropriates about $4 billion annually to Indian and Northern Affairs Canada to fund 
several programs for First Nations communities. The programs include social assistance, education, capital 
facilities, housing, and economic development for approximately 600 First Nations and other Aboriginal groups. 


10.5 In our November 1996 Report, we made recommendations relating to funding arrangements, including 
the FTA, and to the co-ordination of funding across federal departments. We also made recommendations on 


accountability issues and on the suitability of funding arrangements for the needs of the Department and First 
Nations. 


10.6 This follow-up focusses on the Department’s implementation of our 1996 recommendations. 


The Department believes that with respect to redress, the follow-up extends beyond the issues raised in 1996; 
however, it has acknowledged that continuing effort is needed to improve certain aspects of funding 
arrangements, including accountability. 
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Other Audit Observations 


National Defence and 
Health Canada 


The work that led to other audit observations was conducted in accordance with 
the legislative mandate, policies and practices of the Office of the Auditor General. 
These policies and practices embrace the standards recommended by the Public 


Sector Accounting and Auditing Board (PSAAB) of the Canadian Institute of 
Chartered Accountants. 


National Defence and Health Canada 


Non-compliance with conditions and inadequate monitoring with 
respect to the pre-licensing use of an anti-malarial drug 


National Defence participated in a clinical trial of an anti-malarial drug, but did 
not follow the study protocol when the drug was administered to Canadian Forces 
personnel deployed to Somalia. Despite a requirement in the protocol to do so, the 
Department did not obtain consent from the personnel who received the drug, did 
not systematically monitor for efficacy, and did not provide to the study sponsor 
records of the drug’s administration or reports of adverse reactions to the drug. 


Once Health Canada approved the conditions for the clinical trial of the drug, it 
made no attempt to monitor the study to ensure that the trial was adhering to the 
protocol with its reporting requirements and procedures to protect patients’ 
well-being. 


Health Canada is responsible for the regulation and licensing of drugs in Canada. 
An unlicensed drug may be made available only through special measures, such as 
a clinical trial when Health Canada has approved the study design and protocol for 
testing the drug. Some studies test the drug in “real world” conditions and are thus 
a potentially valuable source of information about adverse drug reactions among 
specific populations, efficacy problems in certain environments, and so on. 


Other Audit Observations 


Background 2. As they travel throughout the 
world and are involved in hostile 
ihe Health Canada licenses situations, Canadian Forces members are 


manufacturers to produce and sell drugs 
that have been demonstrated to be safe 
and effective. Only licensed drugs can be 
sold in Canada, except under specific, 
controlled conditions. For example, an 
unlicensed drug may be available through 
the Special Access Program (whereby 
Health Canada approves the sale of the 
drug for a specific patient), or through a 
“clinical trial”, which tests the drug to 
obtain evidence on its safety, dosage and 
effectiveness. A clinical trial is conducted 
under the direction and control of a 
sponsor (usually the manufacturer), but 
the study design and protocol must be 
approved by Health Canada. Once it has 
approved the protocol and received 
information about the investigators 
appointed by the sponsor, the sponsor is 
responsible for conducting the study and 
ensuring that the investigators follow the 
protocol. The sponsor is obligated to 
inform Health Canada of any serious 
adverse reactions (other than those already 
identified) or deaths associated with the 
drug. 


sometimes exposed to health hazards for 
which licensed drugs or other protective 
measures are not available in Canada. 
These may include diseases, such as 
malaria, that are not common in Canada 
or health hazards associated with hostile 
actions such as biological warfare. As a 
result, National Defence must sometimes 
obtain drugs or vaccines through special 
measures. 


3 Canadian Forces members may 
be subject to discipline under the National 
Defence Act if they refuse to submit to a 
treatment, drug or vaccine when ordered 
to do so. National Defence officials told 
us that it is policy not to seek written, 
informed consent when preventive drugs 
or vaccines are prescribed for Canadian 
Forces members during deployments, 
since such consent is often not compatible 
with operational requirements. 


4. Mefloquine is an anti-malarial 
drug that is recommended by the World 
Health Organization and others for use 
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Other Audit Observations 


against some types of malaria that have 
become resistant to other drugs. Although 
licensed in a number of countries since the 
late 1980s, it was not licensed in Canada 
until 1993 and in 1992 was available to 
National Defence only through a Safety 
Monitoring Study. Use of the drug was 
conditional on satisfying the requirements 
of the protocol for the study, including 
obtaining informed consent. 


Sh From November 1990 to early 
1993, mefloquine was available in Canada 
only through an ‘‘open label, 
compassionate access” clinical trial called 
a Safety Monitoring Study, under the 
sponsorship of the drug’s manufacturer. 
The objectives of the study were: 


e to ensure that the Canadian public 
travelling to regions where 
chloroquine-resistant malaria was present 
had access to mefloquine under controlled 
conditions; and 


e tocollect safety data on those 
travellers. 


6. The study was carried out under 
the direction of 21 principal investigators, 
who were medical doctors in travel clinics 
across Canada. The study protocol 
specified the investigators’ 
responsibilities, including keeping 
accurate records on dispensing and 
reporting all adverse drug reactions. It 
stipulated that informed consent was to be 
obtained from all participants and 
specified that “safety data will be 
collected and efficacy will be monitored 
for each subject receiving [mefloquine]”. 
All data and records were to be provided 
regularly to the sponsor (the 
manufacturer). 


I. National Defence participated in 
the study beginning in March 1991, with a 
physician at an Ottawa hospital as a 
principal investigator and a Department 
physician as a co-investigator. 


Issues 


National Defence did not consistently 
follow the protocol for the Mefloquine 
Safety Monitoring Study 


8. From 1991 to July 1992, 96 
National Defence officials travelling to 
Cambodia and Africa were given 
mefloquine under the provisions of the 
Safety Monitoring Study. The Department 
kept records of all but 362 of the 3,500 
mefloquine tablets dispensed; obtained 
consent forms from the travellers; and 
reported to the study sponsor on the 
frequency of adverse effects. 


9, However, National Defence did 
not follow the protocol in the fall and 
winter of 1992-93, when mefloquine was 
dispensed to approximately 900 Canadian 
Forces members before they left for 
Somalia and while there. It did not 
provide the manufacturer with records of 
the drug’s distribution, nor did it obtain 
the consent of those receiving the drug, 
which was not licensed. Canadian Forces 
members were given an oral briefing on 
malaria, mefloquine, and the possible side 
effects, but did not get the written 
documentation given to other Department 
travellers who received the drug. Further, 
even though all supplies of the drug used 
by National Defence were labelled “‘for 
investigational use only”, the Department 
did not systematically monitor efficacy or 
adverse reactions for each person 
receiving the drug, as required by the 
study protocol. It relied instead on a 
disease surveillance system and a periodic 
report of activities to provide any 
indication of side effects or other 
problems with mefloquine. 


10. The manufacturer had identified 
a number of side effects of the drug for 
which patients were to be monitored, 
including gastrointestinal, central nervous 
system and psychiatric disorders. (While 
severe side effects were rare, mefloquine 
was not prescribed for pilots or others in 
occupations “requiring fine co-ordination 
and spatial discrimination, where the 
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sudden onset of dizziness/vertigo can be 
hazardous or life-threatening”.) Although 
69,000 doses of mefloquine had been 
provided to the Canadian Forces medical 
unit in Petawawa in September and 
October of 1992, information on their use 
and on adverse reactions or side effects 
was not reported. Thus, neither the 
manufacturer nor Health Canada benefited 
from information that might have been 
obtained about safety and efficacy of 
mefloquine. 


11. National Defence officials told us 
they did not follow the protocol because 
they believed at the time that they had 
received authorization from Health 
Canada to follow a different set of 
procedures that would not require 
informed consent. However, no such 
authorization was obtained, nor have we 
been provided with any evidence that such 
authority was sought, or even discussed in 
National Defence, with Health Canada or 
with the manufacturer. National Defence 
attributes this confusion to a lack of 
communication between two of its 
directorates. 


12: In an attempt to minimize the 
likelihood of non-compliance with the 
Food and Drugs Act and Regulations, in 
July 1998 National Defence established a 
position with responsibility for all 
regulatory issues relating to unlicensed 
medical products. This position serves as a 
single point of contact between the 
Department and Health Canada’s Health 
Protection Branch. 


Health Canada took no steps to ensure 
the mefloquine study protocol was 
followed by National Defence 


is. Health Canada officials told us 
that although they had approved the 
protocol for the Mefloquine Safety 
Monitoring Study, they took no steps to 
ensure that it was followed. They said that 
monitoring the conduct of the study was 
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the responsibility of the manufacturer, as 
the sponsor of the study. 


14. Health Canada has the right, 
under the Food and Drug Regulations, to 
request copies of the records of a study 
and to terminate the study if it believes it 
is not being conducted properly. However, 
it has no procedures for monitoring the 
conduct of these studies or clinical trials. 


15. We found that National Defence 
was not the only participant in the study 
who failed to provide the manufacturer 
with information about patients and the 
dispensing records of the drug. The 
manufacturer’s final report on the study 
(April 1993), which included the results 
reported by all 21 principal investigators, 
stated that the inability to obtain the actual 
number of patients receiving mefloquine 
made it necessary to estimate the number 
on the basis of pills dispensed. The report 
noted that: 


e 501,424 pills had been shipped to all 
investigators in the study but there were 
records for only 331,695 (66.2 percent); 
and 


e there were an estimated 38,747 
patients, but records for only 25,235 
(65.1 percent). 


16. Mefloquine has been available in 
the U.S. and Europe since the late 1980s, 
and was licensed in Canada in January 
1993, and available on the Canadian 
market in March 1993. However, not until 
October 1994, when the use of the drug by 
Canadian soldiers in Somalia became an 
issue in the media, did Health Canada ask 
the manufacturer for copies of the records 
on the 69,000 doses of mefloquine 
provided to National Defence in 1992. 
The manufacturer did not have any such 
records, although the study protocol called 
for them to be provided to the 
manufacturer every six months; it passed 
the request on to National Defence. When 
the Department could not provide the 
information, Health Canada took no 
action. 
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Conclusion 


ity National Defence did not 
consistently keep essential records or 
follow required procedures to fulfil its 
obligations as a participant in a clinical 
study of an unlicensed drug. As a 
consequence, the integrity of the 
Mefloquine Safety Monitoring Study may 
have been compromised and potentially 
valuable information about the safety or 
efficacy of the drug under “field” 
conditions was not gathered. We have 
noted that, as a result of this situation, 
National Defence has implemented 
measures to increase monitoring and 
improve documentation when using 
unlicensed medical products. 


18. Where unlicensed drugs are 
dispensed through clinical trials, Health 
Canada has the responsibility to review 
and approve the trial design and protocol. 
It needs to assure itself that the conditions 
of clinical trial protocols are met in order 
to preserve the integrity of the process and 
to satisfy conditions set out under the 
Food and Drug Regulations. “Open label” 
trials provide an opportunity to test the 
safety and efficacy of drugs in “real 
world” situations and thus to identify 
potential hazards or problems that may not 
surface under laboratory conditions. They 


represent a potentially valuable source of 
information about the drug and its use. 


National Defence’s response: Despite the 
shortcomings identified in this audit 
observation, cases of potentially lethal 
malaria were prevented and the health 
and safety of Canadian Forces personnel 
were not compromised. At the time of the 
Somalia deployment, mefloquine was 
already licensed in 29 countries, including 
the United States, and the drug had an 
established record of safety and efficacy. 
This record is further supported by the 
recommendations of the World Health 
Organization and by Health Canada’s 
granting of a Canadian licence in January 
of 1993, which coincided with the arrival 
of the main body of Canadian troops in 
Somalia. 


Health Canada’s response: It is Health 
Canada’s policy that the monitoring of 
study protocols rests with the sponsor of 
the clinical trial (usually the 
manufacturer), as well as associated 
institutional research ethics boards, and 
data safety monitoring committees. 
Physicians conducting clinical trials do so 
under provincial/territorial jurisdiction 
and processes are in place to monitor the 
compliance of medical professionals with 
established practice protocols. 


Audit Team 


Assistant Auditor General: Maria Barrados 
Principal: Ronnie Campbell 
Director: Carol Motuz 


For more information, please contact Ronnie Campbell. 
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Chapter 1 


Correctional Service Canada 


Reintegration of Offenders 


The audit work reported in this chapter was conducted in accordance with the legislative 
mandate, policies and practices of the Office of the Auditor General. These policies and 
practices embrace the standards recommended by the Public Sector Accounting and 
Auditing Board (PSAAB) of the Canadian Institute of Chartered Accountants. The 
numbered paragraphs in bold face represent recommendations. 
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Main Points 


iba Correctional Service Canada has made a concerted effort to respond to our 1994 and 1996 observations 
concerning the management of its offender reintegration activities. However, there are some important areas that 
require further improvement. 


12 The Service is now moving in the right direction. It has recently implemented change initiatives in 
several areas. Among them, it has strengthened the ability of national headquarters to direct and co-ordinate 
offender reintegration activities Service-wide; implemented a major initiative to streamline its reintegration 
operations; achieved international recognition for some of its offender rehabilitation programs; and improved its 
ability to measure the results and performance of its reintegration activities. 


Ls Progress notwithstanding, improvement 1s still needed in some key areas: 
¢ more timely acquisition of official documents for initial offender assessment; 
¢ more timely casework preparation to meet the offender’s first parole date; 
e aclear operational strategy for offender employment programs; 
e _ better-quality offender reintegration reports for the National Parole Board; and 


e improved adherence to national standards for frequency of contact with offenders in the community. 


Background and other observations 


1.4 Correctional Service has as one of its main responsibilities the safe reintegration of offenders into the 
community. This entails assessing offender risk and needs; preparing the offender for release into the community; 
reassessing offender suitability for release and making a recommendation to the National Parole Board; and 
providing supervision and programs for offenders in the community until the end of the sentence. 


i sy Overall spending on reintegration has risen by $38 million (13 percent) over the past three fiscal years. 
Correctional Service Canada spends about $329 million or 28 percent of its total expenditures for the reintegration 
of offenders. 


1.6 In 1997-98, there were 13,449 incarcerated offenders in federal institutions and 8,744 offenders in the 
community, most of whom were supervised by Correctional Service parole officers. Until recently, the proportions 
of federal offenders in institutions and under community supervision have remained fairly constant. However, in 
1997-98, the number of offenders supervised in the community increased by about 500. 


see Under the Corrections and Conditional Release Act, there are several different ways that an offender can 
be released into the community: day parole (six months prior to full parole); full parole (at one third of the 
sentence) and statutory release (after two thirds of the sentence). Some offenders will be detained until the end of 
their sentence. 


1.8 A recent change allows offenders serving their first federal sentence who have not been convicted of a 
violent crime or serious drug offence to be released on day parole at one sixth of their sentence (accelerated parole 
review). 


Ji 
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lee Our previous audit work, in 1994 and 1996, identified systemic weaknesses in the Service’s management 
of reintegration activities. Those audits identified concerns in such areas as work standards, quality assurance 
procedures, performance information, implementing basic changes and learning from successes and failures. 

This chapter revisits those issues. 

1.10 As promised to the Public Accounts Committee in April 1998, we reviewed changes made by the 
Service to the custody rating scale and looked at whether the Service had implemented the new offender security 
reclassification instrument. We found that the changes to the custody rating scale reduced overrides with minimal 
impact on the number of escapes. The Service has just implemented a new reclassification instrument, as 
promised. 


1.11 In addition to their institutional security responsibilities, senior correctional officers still do not 
consistently perform their required offender reintegration duties, a necessary input to offender assessment reports 
to the National Parole Board. The Service has undertaken an initiative to address this issue. 


1.12 While the Service has developed a continuum of rehabilitation programs from the institution to the 
community, its ability to deliver these programs to offenders in the community falls short of current needs. 
Research indicates that many intervention programs that deal with offenders’ criminogenic needs are more 
effective when delivered in the community. 


Correctional Service’s responses to our recommendations are included in this chapter. The Service concurs 
with the recommendations made and its responses indicate its commitment to take the necessary corrective 
action. 
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Introduction 


1.13 Correctional Service Canada has 
two main responsibilities — the 
incarceration of offenders and their safe 
reintegration into the community. In 1998, 
the Solicitor General emphasized that 
since offenders come from the community 
and almost all will return there, the best 
way of protecting Canadians is by 
preparing offenders for release. Recently, 
the Commissioner of the Correctional 
Service directed all managers and officers 
to focus on what they could do to get the 
inmates (not just their casework) ready for 
a safe return to the community. 


1.14 In 1997-98 the Service spent 
$329 million or about 28 percent of its 
total expenditures on activities related to 
reintegration of offenders. Overall 
spending on reintegration has risen by 
$38 million (13 percent) over the past 
three fiscal years. 


11S The Appendix provides an 
overview of trends over the last five years 
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in inmate population, admissions and 
releases. From 1994-95 to 1997-98 the 
proportion of federal offenders in 
institutions and under community 
supervision remained fairly constant. 
However, the number of offenders 
supervised in the community increased in 
1997-98 by about S00. 


1.16 Offender admissions have 
declined by 10 percent in the past five 
years. The number of releases increased 
significantly in 1997-98, after declining 
from 1994 to 1997. The data also show 
that offenders are being released at later 
points in their sentences. For example, the 
number of full paroles from institutions is 
down and statutory releases are up. 


1.17 The Corrections and Conditional 
Release Act (1992) provides the National 
Parole Board with the authority for the 
conditional release of offenders on full 
parole at one third of their sentence and 
on day parole six months before the full 
parole date (see Exhibit 1.1). Offenders 
serving their first federal sentence who 
have not been convicted of a violent crime 


Since offenders come 
from the community 
and almost all will 
return there, the best 
way of protecting 
Canadians is by 
preparing offenders 
for release. 


Exhibit 1.1 


Milestones in a Fixed-Term Sentence (Six Years) 


Offender is 1/3 of 2/3 of Expiry of 
sentenced to 6 years sentence sentence sentence 
i (2 years) (4 years) (6 years) 
pee, lbp 
———————————T—. 
| 6 
Accelerated 
i months 
Parole Review* 
ig months Statutory release or 
Day parole continued detention 
eligibility 
(18 months) 
Full parole eligibility 


* Accelerated Parole Review: All offenders serving their first federal sentence who have not been convicted of a violent 


crime or serious drug offence must have their parol 
Accelerated Parole Review process and criteria. Fo 


e eligibility reviewed by the National Parole Board using the 
r those offenders, day parole eligibility is set at one sixth of the 


sentence rather than six months prior to the parole eligibility date. 
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or serious drug offence can be released on 
day parole at one sixth of their sentence 
(Accelerated Parole Review), unless there 
are reasonable grounds to believe they 
will commit a violent offence before the 
end of their sentence. 


1.18 Most offenders are entitled by 
law to be released (statutory release) after 
serving two thirds of their sentence, if the 
offender is not on parole at that time. A 
small number of offenders will be 
detained in prison until the end of their 
sentence, based on a recommendation 
from Correctional Service to the National 
Parole Board. The Board detains offenders 
when it is likely that, if released, they will 
commit an offence involving death or 
serious harm, a sexual offence against a 
child, or a serious drug offence before the 
end of their sentence. 


1.19 Under the Act, the purpose of 
conditional release is to contribute to 
public safety by releasing offenders at a 
time and in a manner that increases their 
chance for successful reintegration into 
the community. There has been a 
significant decrease in the number of 
offences committed by offenders while 
under the Service’s supervision in the 
community, that is, a 37 percent decrease 


in offender revocations for new offences 
since 1993—94. Revocations for violent 
offences also declined over the same 
period, from 210 to 161 (see Exhibit 1.2). 


The offender reintegration process 


1.20 Correctional Service uses a case 
management process to manage the 
reintegration of offenders (see 

Exhibit 1.3). The current process consists 
of a number of stages, including: 


e obtaining official documents 
required for assessing the security risk and 
the needs of the offender; 


e assessing offenders when they enter 
the federal corrections system to identify 
the factors that led to their criminal 
behaviour and developing a correctional 
plan to address them; 


e assessing whether participating in 
programs or other interventions has helped 
reduce the risk that an offender will 
commit another offence after release; 


e making recommendations to the 
National Parole Board on the offender’s 
suitability for release to the community; 


e releasing the offender on parole by 
the National Parole Board, or by law after 
serving two thirds of the sentence 


Exhibit 1.2 Revocations for 
Violent Offence 1993-94 1994-95 1995-96 1996-97 1997-98 
Federal Offenders Under Murder iG 15 10 9 
Community Supervision 
Charged With Violent Attempted Murder 9 15 Uf 9 
Offences Sexual Assault 43 49 22 31 23 
Fiscal year ending 31 March Major Assault Hi 25 21 28 34 
Hostage Taking 4 5 3 l l 
Unlawful Confinement 9 3 5) 4 6 
Robbery 102 Wis. 71 90 79 
Subtotal 210 224 152 171 161 
Other Revocations 
for Offence 1,353 1,065 999 838 819 
Source: Correctional ; 
SonieeGan ade Total 1,563 1,289 1,151 1,009 980 
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(statutory release) or at warrant expiry Rehabilitation Programs for Offenders; 
(end of sentence); and 1994 Chapter 18, Supervision of 
Released Offenders. As a result, this audit 
covered the major aspects of offenders’ 
reintegration, from their intake into 
Correctional Service Canada until the end 
of their sentence. 


e providing supervision, further 
programs and assessment in the 
community until the end of the sentence; 
and 


* reporting to the National Parole 1.22 Our objectives in this audit were: 
Board on cases where circumstances 
elevate the risk presented by the offender, 
and thus warrant review by the Board. 


e to determine the extent to which 
Correctional Service has acted on our 
earlier recommendations; and 


Focus of the audit e to assess the extent to which the 
Service’s changes in the management of 
offender reintegration have contributed to 
sustainable improvements. 


1.21 The focus of this audit was to 
follow-up and re-audit key observations 
and recommendations made in our 1996 
Report Chapter 30, Reintegration of 1.23 The audit also included follow-up 
Offenders; 1996 Chapter 10, audit work on the Custody Rating Scale 


Exhibit 1.3 


The Offender Reintegration Process 


Otfender: pire Institution Community 
Unit 
Acquisition of parole denied National parole granted 
Official Information | | Parole Board ry 
(NPB) 
| Hearing 
y = 


Correctional 
Plan! 


Offender Intake 
Assessment 


Community | Correctional 
Supervision | Programs 


| 


SS 


ie Continuing 
Institutional 
: Further 
rograms Assessment 


’ | 


Case Preparation 


—— = 4 Statutory 
Release 


Sentence Begins Sentence Ends 


Note 


! As a result of Operation Bypass implemented in February 1999, correctional plans are 
completed in the Service’s Offender Intake Units and amended if required to deal with 
changes in the factors affecting the offender’s criminal behaviour. Source: Office of the Auditor General 
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The Service has 
established its 
minimum information 
needs. 


1-10 


and Security Reclassification instrument 
(components of offender risk assessment) 
in response to a request by the Public 
Accounts Committee in April 1998. 


1.24 As in our previous audits, this 
audit did not include women offenders or 
issues specific to Aboriginal offenders. 
Further details on the audit can be found 
at the end of the chapter in About the 
Audit. 


Observations and 
Recommendations 


Offender Assessment and Case 
Management 


1.25 In September 1996, Correctional 
Service established a Reintegration 
Taskforce to devise short-term and 
long-term strategies for improving the 
management of offender reintegration 
activities. As a result of the Taskforce’s 
recommendations, in 1997 the Service 
undertook three national reviews of major 
components of the offender reintegration 
process (offender intake, institutional case 
management and community case 
management). 


1.26 Taskforce and national review 
recommendations culminated in the July 
1998 approval of a major change initiative 
called Operation Bypass. The objective of 
this initiative was to reduce duplication of 
tasks and information; consolidate 
offender reports; and strengthen 
communication and co-ordination between 
institutional and community parole 
officers. Several regions piloted aspects of 
Operation Bypass in 1998. The Service 
implemented it in all regions in February 
1999. This chapter’s discussion of our 
audit observations outlines the details of 
this initiative and its potential impact on 
each aspect of the reintegration process. 


Standard for official information is 
defined 


1.27 In 1996 we indicated that 
Correctional Service had difficulty 
obtaining official documents from 
provinces and municipalities, including 
police reports, Crown briefs and judges’ 
reasons for sentence. The lack of such 
information can affect the quality of 
rehabilitation planning, slow the process 
of preparing offenders for parole and 
affect the quality of National Parole Board 
release decisions. Correctional Service 
internal investigations showed that 
missing documentation had been a 
contributing factor in some incidents 
involving violent offences committed by 
released offenders. We recommended that 
the Service decide on its minimum 
information needs and, with its partners in 
the criminal justice system, speed up the 
acquisition of relevant information. 


1.28 As part of the Operation Bypass 
initiative, Correctional Service approved a 
policy requiring the following official 
documents as a prerequisite to assessing 
the offender’s risk and needs, leading to 
the development of the offender’s 
correctional plan: 


e an official version of the offence 
(court documents, police report or 
pre-sentence report); 


e the offender’s criminal history from 
police records; and 


e the Post-Sentence Community 
Assessment — a document prepared by 
Correctional Service community parole 
officers that includes such elements as 
offender employment history, family 
relations, behavioural problems and 
victim information. 


Document acquisition standards are not 
being met 


1.29 In 1997, the Service conducted a 
national review of its intake assessment 
operations and procedures. It found that 
not all required documents were being 
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obtained before Offender Intake 
Assessments were completed. Among 
other things, the review reported regional 
variations in the timely acquisition of 
documents; uneven quality of police 
reports; and difficulty accessing Crown 
files. In addition, the Post-Sentence 
Community Assessments completed by 
parole officers in the community were of 
poor quality. The review concluded that 
the quality of the Offender Intake 
Assessment “is highly dependent on the 
experience of the Parole officer who 
completed it and on the information 
available at the time the report was 
completed”. 


1.30 From the date of admission to the 
Intake Unit, Correctional Service policy 
allows a maximum of 56 days to complete 
the Offender Intake Assessment. To ensure 
that the standard is met, the parole officer 
needs to receive the necessary documents 
promptly. The Service compiles data 
showing when many of these documents 
(police reports, Post-Sentence Community 
Assessments and judge’s comments) are 
received.’ 


13k We examined a sample of these 
data to assess the level of improvement 
since our last audit; we found that 
documents are arriving sooner. However, 
about one quarter of the documents are 
still not received within the 56-day time 
frame (see Exhibit 1.4). We also found 
that parole officers were completing many 
Offender Intake Assessments before 
receiving either the desired police reports 
or the Post-Sentence Community 


I 
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1.33 In the four regions that we 
visited, the Service has memoranda of 
understanding and agreements with 
provincial governments and municipalities 
for the timely acquisition of required 
official documents. Regional intake units 
are using a variety of techniques to speed 
up acquisition of documents. For example, 
all regions have some kind of system to 
track and follow up specific documents. 
Some regions have made direct contact 
with each external agency and agreed on 
document turnaround targets. They are 
also pursuing new ways to transmit 
documents electronically. 


1.34 Although the timeliness of 
document acquisition continues to 
improve, much remains to be done. Under 
Operation Bypass, the need for 
improvement will increase as the Service 
shortens its time frame for completing 
both the Offender Intake Assessment and 
the Correctional Plan. While Correctional 
Service Canada is resolving many of the 
interagency issues with its partners in the 
Justice system, it needs to further improve 
the results and level of performance. 


Although the 
timeliness of 
document acquisition 
continues to improve, 
much remains to be 
done. 


135 Correctional Service Canada 
should seek ways to further improve the 
timeliness of its acquisition of required 
information on offenders. 


Exhibit 1.4 


Official Information Available Within 56 Days After Date of Offenders’ Admission* 


(percentage of cases) 


Assessments (or both). Pac Sep lonce | 
Community 

1.32 Correctianal Service hace June to August Police Reports Judges’ Comments Assessments 

. = 5 
standard of 30 to 45 days (depending on 1996 64% 58% 60% | 
the length of the offender’s sentence) to ee ee 689% 70% 
prepare the Post-Sentence Community 
Assessment, a report generated within the 1998 72% 16% TA% 

+— = a —_——— = ——— ae 


Service itself. Correctional Service data 
indicate that between July and September 
1998, less than 40 percent of the 
assessments met this standard. 


* For male offenders admitted under warrant of committal. 


Source: Office of the Auditor General of Canada using 
data obtained from Correctional Services Canada 
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Exhibit 1.5 


Correctional Service’s response: The 
Service agrees with this recommendation 
and is actively working to address the 
issues. In the area of collection of 
information from other agencies, 
Correctional Service has no authority to 
compel those agencies to provide the 
required information in a timely manner. 


The number of offenders incarcerated 
beyond their first parole eligibility date 
remains high 


1.36 Failure to prepare offenders in 
time for their first parole eligibility date 
may adversely affect their chances for safe 
reintegration. It also has cost implications, 
since federal statutes require Correctional 
Service to maintain offenders in the least 
restrictive level of custody. However, the 
protection of society must remain the 
paramount consideration. 


1.37 Correctional Service data 
indicate that as of November 1998, 

21 percent (2,782) of incarcerated 
offenders had not had a National Parole 
Board release hearing and were past their 
eligibility date for either day parole or full 
parole. In some of the cases this may be 
acceptable — for example, offenders may 
choose to waive or postpone a hearing for 
their own reasons. In other cases, 
however, the reason may be the Service’s 
inability to prepare the offender’s case in 
time. 


Scheduled Parole Board Hearings Waived or Postponed 


(numbers and percentages*) 


Total Waived 
Day Parole Full Parole or Postponed 
1 
1995—96 2,371 (23%) 6,529 (43%) 8,900 
1996-97 1,981 (23%) 6,441 (46%) 8,422 
1997-98 1,444 (14%) 4,617 (38%) 6,061 


* percentages of all hearings that were scheduled 


Source: Correctional Service Canada 


1.38 Each year a significant number of 
scheduled hearings before the National 
Parole Board are waived or postponed by 
the offender (see Exhibit 1.5). In 1997-98, 
14 percent (1,444) of all day parole 
hearings and 38 percent (4,617) of all full 
parole hearings were waived or postponed. 
Although these percentages dropped 
between 1996-97 and 1997-98, the 
overall number still remained high. 
Correctional Service has yet to analyze 
why so many hearings are waived or 
postponed. 


1.39 Furthermore, the National Parole 
Board can defer a parole hearing by 
means of an administrative adjournment. 
The Board does so when it feels that 
information on the offender is incomplete 
or the offender’s case has not been 
prepared sufficiently for a hearing. 
National Parole Board officials estimated 
that they adjourned about seven percent of 
all full parole hearings due between April 
and September 1998. 


1.40 Correctional Service Canada 
should regularly analyze the reasons for 
the number of offenders incarcerated 
beyond their first parole eligibility date 
in order to minimize late casework as a 
reason for cancelling parole hearings. 


Correctional Service’s response: The 
Service agrees with the recommendation 
and while it monitors the number of 
waivers, it will conduct more specific 
analysis on the reasons for waivers. While 
the rate of waivers sometimes appears 
high, it is still the offender’s right to apply 
for or waive reviews for any form of 
conditional release. The issue of waivers 
will be further discussed in the context of 
the Corrections and Conditional Release 
Act review. 


Completing offender programs in time 
for first parole date remains a challenge 


1.41 As we did in 1996, in this audit 
we reviewed how long it takes to process 
and program offenders serving two- to 
three-year sentences for their first day 


a ee a ee eee 
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parole eligibility date. These offenders are 
eligible for day parole in 189 days, on 
average approximately six months after 
sentencing. Because the National Parole 
Board requires all relevant case 
documents at least 21 days before a parole 
hearing, the Service would have to 
complete prescribed programs, prepare 
case documentation, and evaluate the 
offender within 168 days of sentencing. 


1.42 Processing the offender through 
the intake assessment unit is the first step 


i 
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in the reintegration process. In 1995—96 it 
took an average of 87 days after 
sentencing to complete an Offender Intake 
Assessment (see Exhibit 1.6). Between 
January and April 1998, this was down to 
71 days. As noted earlier, the Service has 
established the maximum time frame to 
complete Offender Intake Assessments at 
56 days from admission. Correctional 
Service data indicate that about one third 
of offenders with two- to three-year 
sentences are not assessed in time to meet 
that standard. 


About one third of 
offenders with two- to 
three-year sentences 
are not assessed in 
time. 


Exhibit 1.6 


Average Time to Complete Case Management Process 


1995-96 and January-April 1998 data are based on a sample of male 
offenders with 2- to 3-year sentences admitted to federal institutions during 
these time frames, with sample sizes of 1,800 and 618 offenders respectively. 


1995-96 
Offender arrives at 
Sentence federal institution Offender Intake Correctional Plan 28-day NPB* Day parole 
begins (intake unit) Assessment completed completed requirement eligibility date 
One 24 87 183 days 
— Complete prescribed programs 
— Evaluate benefits 
— Prepare case for National Parole Board 
January-April 1998 
Offender arrives at Offender Intake 
Sentence federal institution Assessment Correctional Plan 21-day NPB* Day parole 
begins (intake unit) completed completed requirement eligibility date 
0 26 108 168 189 days 
| 


* National Parole Board 


— Evaluate benefits 


— Complete prescribed programs 


— Prepare case for National Parole Board 


Source: 1995-96 data are from the 1996 Report of the Auditor General (Chapter 30). January to April 1998 data were obtained from 
Correctional Service’s offender management system. 
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1.43 The second step in the process is 
the completion of correctional plans. The 
offenders are then moved from intake 
units and placed in penitentiaries, where 
they can participate in prescribed 
rehabilitation programs. In 1995—96 it 
took 137 days to complete a correctional 
plan. Between January and April 1998, 
this was down to 108 days. The Operation 
Bypass standard is now 70 days. 


1.44 We estimated in 1996 that, on 
average, three rehabilitation programs 
were prescribed for low-risk/low-need 
offenders, and the shortest possible time in 
which they could complete the three 
programs was 105 days. In 1996, only 
19 days were available for offenders to 
complete programs before their earliest 
eligibility date for day parole. Although 
this had increased to 60 days by 1998, 
there was still not enough time to 
complete prescribed programs by the 
earliest possible parole date. 


1.45 Correctional Service expects that 
Operation Bypass will address this 
problem by having the correctional plan 
done in the Intake Unit and the 
community strategy completed before the 
offender’s release. As a result, the Service 
forecasts that the time available for 
programs, assessments and report 
preparation before the National Parole 
Board’s cut-off date will increase from the 
current 60 days to 106 days. 


1.46 Correctional Service’s Canada 
should further improve the efficiency 
and timeliness of its offender case 
management and report preparation to 
meet its approved time standards. 


Correctional Service response: The 
Service agrees with this recommendation 
and has taken steps to improve efficiency 
and timeliness. The Auditor General has 
noted improvements to that effect. The 
results of reintegration operations will be 
regularly reviewed to monitor progress 
and determine if adjustments are required. 


Parole officer training is improving 


1.47 An effective offender 
reintegration process requires both 
competent parole officers and the 
involvement of senior correctional 
officers. In 1996 we found that training for 
parole officers was inadequate, and 
correctional officers were not fulfilling 
their case management responsibilities. 


1.48 We noted that while parole 
officers received only eight days of 
orientation training, many did not begin 
their initial training until after starting 
their job. Our follow-up work indicates 
that the amount and variety of training 
available to parole officers has increased 
to some extent. Ten days of orientation 
training is now a formal requirement for 
new officers. In 1996-97, 60 new parole 
officers received orientation training in 
reintegration; this rose to 232 in 1997-98. 
The Service indicates that about 200 new 
officers have been trained from April 1998 
to January 1999. 


1.49 Officers’ participation in such 
non-mandatory courses as risk assessment 
and quality assurance has also increased. 
In general, experienced officers have 
received less reintegration training 
annually than new officers. Plans are 
under way to design and provide refresher 
training for experienced officers in the 
near future. 


1.50 The Service is now providing 
training for the implementation of 
Operation Bypass to about 4,000 staff. In 
addition, existing orientation training for 
parole officers is being redesigned as a 
proposed program of at least 20 days. 


Senior correctional officers are still not 
fulfilling their case management 
responsibilities 


1.51 The offender reintegration 
process requires teamwork between parole 
officers and senior correctional officers 
(senior staff with security responsibilities) 
in institutions. Their collaboration and 
exchange of information is crucial for 
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them to understand changes in offenders’ 
attitudes and behaviours and to prepare 
assessment reports for the National Parole 
Board. 


1.52 In 1996 we observed that senior 
correctional officers were not consistently 
performing the case management duties 
required of them by Correctional Service 
policy, despite management’s continuing 
awareness of the problem. These officers 
are expected to, among other things, 
complete clear, concise reports for case 
management, inform colleagues of 
significant incidents or behavioural 
changes of inmates, and participate in the 
assessment and planning of case 
management for inmates. 


1.53 Our audit work in four regions 
found a wide variance in the involvement 
of senior correctional officers in offender 
case management and reintegration, and 
in their support for it. Senior correctional 
officers continue to perform these duties 
inconsistently. Parole and senior 
correctional officers we met cited a lack 
of time and training, particularly in such 
areas as computer skills, as reasons why 
correctional officers have not adequately 
performed their duties in offender case 
management. 


1.54 Correctional Service has recently 
revised a Unit Management 
Accountability Matrix to further clarify 
national policy in this area. The Matrix 
outlines in detail the responsibilities of 
senior correctional officers and parole 
officers in light of the operational changes 
required by the Operation Bypass 
initiative. Regional management has 
agreed to these changes and plans to 
implement the matrix in early 1999. 


1:55 Correctional Service Canada 
should ensure that senior correctional 
officers perform their offender case 
management duties, as required by 
policy. 


Correctional Service’s response: The 
Service agrees with this recommendation. 
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The decision in April, 1998 to recruit 
1,000 additional correctional officers was 
made in large part so that senior 
correctional officers can fulfill their case 
management duties. The revised Division 
of Responsibilities Matrix was 
implemented in February, 1999. In the 
past, the attention and commitment 
required on the part of CSC management 
to deal with this issue was lacking, and 
the Service is resolved to do better. This 
will require close and regular monitoring 
of performance against the new matrix, 
and firm and fair action regarding both 
compliance and non-compliance with it. 


We found a wide 
variance in the 
involvement of senior 
correctional officers in 


offender case 
Offender Programs management and 
1.56 The Service has a wide range of reintegration. 


rehabilitation programs to treat offenders 
in order to reduce offender risk. These 
programs can be broken into two broad 
categories: intervention and employment. 
Intervention programs have been 
developed to address characteristics 
related to criminal behaviour. They deal 
with such areas as treatment for sex 
offenders, substance abuse, family 
violence initiatives and living skills. 
Employment programs are designed to 
enhance the offender’s employability on 
release, and also serve to keep them busy 
with meaningful work during 
incarceration. They include vocational 
training, prison industries (CORCAN), 
adult education and Institutional Services 
(areas such as the kitchen, laundry, 
institutional maintenance and stores). 


1,57 In 1996 we concluded that both 
intervention and employment programs 
lacked a management framework by 
which senior management could reassess 
and strategically reallocate funds. We had 
expected that the Service would have 
mechanisms in place to monitor both the 
effectiveness and the costs of all 
programs. This, in turn, would enable 
management to verify progress toward 
strategic objectives and to make any 
necessary adjustments. 
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Intervention programs 
are being 
internationally 
accredited. 


Exhibit 1.7 


Rehabilitation Programs 
Expenditures 


($ millions) 


Source: Correctional Service 
Canada 


Cost information for rehabilitation 
programs has improved 


1.58 Our previous audit indicated that 
Correctional Service had limited 
information on the costs of its 
rehabilitation programs. In April 1998, 
national headquarters implemented a new 
system for capturing the costs of offender 
programs. This will improve the Service’s 
ability to collect and assess program costs 
and cost trends. 


1.59 Exhibit 1.7 compares most 
rehabilitation program expenditures for 
1994-95 and 1997-98. Total expenditures 


were $110 million in 1994-95 and 

$126 million in 1997-98. The costs of 
intervention programs increased by 

$6 million (18 percent) between 1994—95 
and 1997-98. Costs of employment and 
education programs were $8 million 
greater (22 percent) in 1997-98 than in 
1994-95. 


A framework is in place for evaluating 
intervention program effectiveness 


1.60 For the past 10 years, 
Correctional Service has made a concerted 
effort to provide intervention programs 
that have a potential for reducing 


Intervention Programs! 


ee el 


| 1994-95 1997-98 Increase 


Sex offender 9 10 1 
Substance abuse 8 8 - 
Locally designed 3 4 1 
Living skills 5 Bl - 
Administration 2 5 D 
Other? 6 8 2 
Subtotal 34 40 6 
a 

Employment and Education a ee 

Basic adult education 13 14 iid i 
Vocational training 4 4 = 
Administration for education and vocational training = 
CORCAN (prison industries)3 16 23 

Subtotal* 36 44 8 


Other Activities 


Social and cultural programs 


fe ee ei 


(visits, hobbycraft, recreation) 14 15 ] 

Inmate pay 20 20 — 

Chaplaincy 6 if l 

Subtotal 40 42 2 

Total 110 os 126 hae 16 
Notes 


| 


administration. 


ww 


Except for CORCAN, figures above exclude regional and national headquarters management and 


Aboriginal programs, family violence programs, community psychological counselling. 


For the purpose of this exhibit, CORCAN’s costs consist of a training and correctional fee paid to 


it by the Service ($18 million in 1997-98) and CORCAN’s use of borrowing authority ($7 million 
in 1997-98). They exclude interest ($1 million in 1997—98) on borrowings paid to the 
Consolidated Revenue Fund by CORCAN. Its annual financial statements are the best source of 
information to understand its overall financial results. 


The training expenditures associated with offenders employed in institutional and offender 
services (i.e. kitchen, groundskeeping) are not included. 
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recidivism. These programs are delivered 
mainly in institutions, although some 
continue after the offender’s release into 
the community. 


1.61 Under the new policy on 
rehabilitation programs, each program that 
targets offenders’ criminogenic factors 
and contributes to their rehabilitation shall 
have clearly articulated objectives, criteria 
for the selection of participants, a process 
for evaluating participants’ progress, and a 
process for measuring program 
effectiveness. Programs are to be 
delivered as they were designed, and by 
trained, qualified staff. 


1.62 Using these criteria for programs, 
Correctional Service is now setting up 
panels of internationally recognized 
experts in both corrections and specific 
subject matters (such as substance abuse); 
the panels provide independent 
accreditation of programs as state-of-the- 
art. Programs will be reaccredited every 
five years, with site accreditations on a 
three-year cycle. If a program at a local 
site is not accredited, regions are being 
advised to upgrade it so that it can meet 
accreditation standards or to discontinue it 
and switch to an accredited national 
program. This is to ensure that all 
programs being delivered are equally 
effective. 


1.63 Exhibit 1.8 summarizes the 
programs the Service has submitted for 
accreditation to date, as well as the 
results. Some programs have been 
accredited; others have been accredited 
with conditions that are to be satisfied 
within a stated time frame; and some have 
not been accredited and must be 
resubmitted at a future date. As one of the 
additional benefits of the accreditation 
process, the Service has identified an 
overprescription of institutional substance 
abuse programs. This is now being 
addressed. 


1.64 The information used to present 
programs to the accreditation panel will 
provide the basis for subsequent 
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evaluations of the programs’ impact on 
inmate recidivism. An evaluation of 
substance abuse programs 1s currently 
being completed. These evaluations, 
combined with information on actual 
program costs, should enable management 
to make any adjustments needed to fulfil 
intended policy direction. 


More intervention programs in the 
community are needed 


1.65 According to research, many 
intervention programs that deal with 
offenders’ criminogenic factors are more 
effective when delivered in the 
community setting than in the institution. 
In 1996, we stated that the Service had not 
established a continuum of programs from 
the institution to the community to 
properly support offenders in their 
transition to the community. 


1.66 Correctional Service is seeking a 
more balanced approach to institutional 
and community programming for 


Correctional Service is 
still seeking a more 
balanced approach to 
institutional and 
community 
programming for 
offenders. 


Exhibit 1.8 


Status of Intervention Programs Submitted for Accreditation 


| 
Programs Status | 
Substance Abuse e 2 national programs accredited | 
e a number of local programs still to be reviewed | 
| 
Living Skills e 2 national programs accredited with conditions | 
to be met (anger and emotion management and 
cognitive skills) 
e more national programs to be reviewed 
Sex Offender e | local program fully accredited 
e 2 local programs not accredited, to be revised 
and resubmitted 
e next panel will review 4 more programs 
Offender Violence e next panel will review 4 programs 
Family Violence e a panel to review these programs is planned for 
2000 
Other Programs — i.e. e need for accreditation is being assessed 
ce Ditenders e if needed, strategy will be developed in 
Aboriginal Offenders ‘ areal ; 
consultation with stakeholder groups 
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offenders. The objective is to deliver in 
institutions only correctional programs 
needed to reduce the offender’s risk upon 
release. Community programs include 
maintenance and “booster” programs that 
build on institutional programs, as well as 
programs better suited to delivery outside 
the institution. The Service has developed 
intervention programs in the community 
that address substance abuse, treatment for 
sex offenders, cognitive skills, violence 
prevention and family violence. 


1.67 Implementation of this change in 
approach has begun. Spending on offender 
programs in the community remained at 
generally the same level from 1994-95 
($8.2 million) to 1997-98 ($8.4 million). 
But it decreased as a percentage of the 
total spending on all intervention 
programs, which grew from $34 million to 
$40 million over the same period. In its 
National Capital Accommodation and 
Operations Plan for 1999-2000, the 
Service acknowledges that its community 
organization is not adequately structured 
to deliver correctional programs and its 
community program resources are 
insufficient to meet current needs. 


1.68 As one of the requirements for 
program accreditation, the content of 
institutional programs is to be continued 
and reinforced in the community. The 
Service has determined that some released 
offenders with substance abuse problems 
are not receiving required follow-up 
treatment programs in the community. 
This came to light during the current 
program accreditation process and is now 
being addressed. 


1.69 There are obstacles to effective 
delivery of community programs. 
Typically, the Service has delivered such 
programs to groups of about 10 offenders. 
Often there are not enough offenders with 
similar programming needs in any given 
area (except in large urban centres) to 
make such groups viable. The Service is 
considering alternative approaches with 
smaller group sizes (down to 


four offenders) in smaller cities and 
towns, and individual consultations 
delivered by trained community parole 
officers for offenders released to rural 
communities. These changes will make 
the delivery of programs more costly on a 
per capita basis but nevertheless more 
cost-effective than continued 
incarceration. 


1.70 In 1996 we also noted specific 
areas where offenders with identified 
needs were not receiving required 
programs in the community. The Service 
does not yet have data to show whether 
this gap has been addressed. It has 
modified its information systems to 
provide such data in the future. However, 
Correctional Service indicated that it now 
requires that parole officers in institutions 
plan programs and interventions to 
manage risk in the community. 


1.71 Correctional Service Canada 
should ensure that there is an 
appropriate balance between 
institutional and community 
intervention programs. 


Correctional Service’s response: The 
Service agrees with this recommendation. 
CSC has recognized the problems and 
adjusted the National Capital 
Accommodation and Operations Plan to 
secure additional resources in 1999-2000 
to increase program capacity in the 
community. This will improve the balance 
between institutions and the community. 


An overall strategy for employment 
programs is needed 


1.72 Research indicates that the risk of 
re-offending is much greater among 
offenders with unstable employment 
patterns than those with a stable 
employment history. The Service provides 
employment programs such as adult basic 
education, vocational training, CORCAN 
(prison industries), Institutional Services 
such as kitchen, laundry, maintenance and 
stores, and services to offenders such as 
recreation and library services. Exhibit 1.9 
provides the results of a Correctional 
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Service national survey that estimated 
offenders’ participation in these 
employment areas in 1997. As well, the 
importance of keeping offenders busy in a 
meaningful way in the institutional setting 
should not be minimized. 


1.73 In 1996 we stated that 
Correctional Service did not have a 
coherent strategy for employment 
programs. Resources dedicated to these 
programs were not well managed and the 
Service did not have the ability to make 
any needed trade-offs among all 
employment programs, including 
CORCAN. 


1.74 Since our 1996 audit, 
Correctional Service has undertaken a 
number of studies to examine the state of 
all employment programs. Some critical 
themes have emerged from this work: 


e The cost effectiveness of 
employment programs in reducing 
recidivism is not known. 


e Offenders’ employment needs are 
not consistently well identified. 


e Inmates are not systematically 
assigned to work based on their needs. 


e Parole officers do not generally see 
work as a valid correctional intervention. 


e There is no accountability for the 
extent to which an inmate’s needs are 
addressed. 


e There is no continuum between 
institutional and community employment. 


1.75 These studies have recommended 
that the Service adopt practices to ensure 
that the validity and correctional value of 
employability programs match their costs, 
as it has for intervention programs. Senior 
management recently approved the 
creation of two short-term task groups to 
explore these issues of offender 
employment. 


1.76 In their January 1999 report to 
the Executive Committee, the task groups 
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made several key recommendations 
related to offender employment. Some key 
messages in this report included the 
following: 


e inmate employment needs should be 
better assessed and entered consistently in 


the correctional plan; : * 
j Uncertainty remains 


about the role of 
employment programs 
in effective 
reintegration. 


e work assignments should be 
meaningful and should be seen as a 
correctional intervention and managed 
accordingly; 


e Grade 12 should be the new 
education norm for offenders (previously 
Grade 10); and 


e inmate career planning, job search 
and community employment assistance 
need improvement. 


1.77 The Service continues to spend a 
significant portion of its rehabilitation 
program expenditures on employment 
programs. However, little information is 
available on the cost effectiveness of 
various types of employment programs. 
Uncertainty remains about the relative 
priority of employment programs and their 
role in effective reintegration of offenders. 


1.78 Correctional Service has invested 
much of its effort over the past three years 


Exhibit 1.9 


Community Work 
(453) 


CORCAN 
/ (1,787) 


Offender Participation in 


Vocational Training Employment Programs 


(1,354) \_ 
Estimate as of September 1997 


(full-time equivalents*) 


Basic Adult Institutional 
Education Services 
(2,708) (3,949) 


Services to Offenders 
(922) 


* A full-time equivalent represents one offender working 
full-time for one year. Two offenders each working half 
a year represent one full-time equivalent. 


Source: Correctional 
Service Canada 
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Many institutions still 
have no clear process 
for matching 
offender’s employment 
needs with programs. 


in the institutional offender intake and 
case management processes. It has made 
progress in setting a strategic direction for 
intervention programs and establishing an 
accreditation process to ensure their 
quality. However, a clear operational 
strategy for employment programs is 
needed. 


1.79 In its strategic planning of 
offender programs, Correctional 
Service Canada should clarify the role 
that employment programs have in 
offender reintegration and should 
manage and fund these programs 
accordingly. 


Correctional Service’s response: The 
Correctional Service recognizes that 
employment assignments must be treated 
in the same manner as other correctional 
programs. With effective planning and 
scheduling of all interventions during the 
initial assessment process, employment 
assignments should complement other 
correctional interventions. CSC is 
confident that the work currently under 
way will effectively address the 
management of employment programs 
within the Service. 


Offenders with employment needs are 
not targeted for suitable programs 


1.80 CORCAN provides a good 
illustration of the need for clear direction 
in employment programs. In 1996 we 
examined CORCAN in some detail 
because of its unique status in the Service 
as a special operating agency, and the 
level of its expenditures. We followed up 
in this audit with the same focus, namely, 
CORCAN’s ability to train properly the 
inmates who need training most, and its 
financial self-sustainability. 


1.81 CORCAN’s charter states that 
one of its main objectives is to provide 
offenders with work-related training and 
work experience in accordance with needs 
identified in their correctional plans. In 
1996, we reported that CORCAN had not 
targeted offenders who lacked 
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employability skills. In the majority of 
cases we examined, neither the files nor, 
more specifically, the offender’s 
correctional plan indicated that the 
offender needed the employability training 
provided by CORCAN. 


1.82 Offenders are evaluated during 
Intake Assessment to identify the level of 
their employment needs as considerable, 
some or no needs. In this audit, we 
examined the employment needs of a 
random sample of 1,526 offenders 
working for CORCAN in institutions at 
various security levels (see Exhibit 1.10). 
We found that the majority of these 
offenders had at least some employment 
needs. We also noted that offenders with 
considerable employment needs 
constituted less than a quarter of the 
CORCAN work force in minimum and 
medium security institutions. In maximum 
security institutions, about half of the 
offenders working for CORCAN had 
considerable employment needs. 


1.83 In the regions we visited for this 
audit, we found that many institutions still 
have no clear process for matching 
identified offender needs for 
employability skills with employment 
programs. While inmates are encouraged 
to participate in employment programs, 
their participation is voluntary. Inmates 
are generally left on their own to look for 
work in the institution. CORCAN 
instructors assess their need for workers 
and seek those inmates best suited to the 
jobs. Many CORCAN officials do not 
easily have access to correctional plans. In 
two institutions, we compared the 
employment needs of offenders working 
for CORCAN with those of offenders 
working in other institutional jobs. We 
found no significant differences in levels 
of need. 


1.84 To be successful, an employment 
program needs to deliver the right kind of 
training to the most needy offenders, and 
for a long enough period to make a 
difference. However, offenders tend not to 
work for CORCAN for very long. We 


Report of the Auditor General of Canada — April 1999 


examined the employment records of 
offenders released between April and 
October of 1998 who had worked for 
CORCAN. We found that two thirds of 
these offenders had worked for CORCAN 
for less than six months over their entire 
period of incarceration. About half of their 
job assignments were for three months or 
less. Neither the Service nor CORCAN 
has studied the minimum length of work 
experience needed to make a difference in 
an offender’s employability. 


CORCAN’s self-sustainability is still in 
question 


1.85 As a special operating agency, 
CORCAN has a continuing goal to attain 
financial self-sustainability. The Service 
pays CORCAN a training fee of about 
$18 million per year for the employment 
services it provides to inmates. When 
CORCAN became a special operating 
agency, it was given access to a 

$45 million revolving fund to be used to 
acquire capital and cover any operating 
losses. Approximately $12 million 
remains ‘in its revolving fund. 


1.86 In 1996 we concluded that 
CORCAN was not meeting its goal of 
financial self-sustainability. Between 
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1994-95 and 1997-98, it increased the 
employment of offenders by 21 percent 
and lowered its cost per offender by 

29 percent. Over the same period, it had 
cumulative operating losses totalling 

$12 million (see Exhibit 1.11). CORCAN 
projects further losses over the next 

two years. 


1.87 When it approved CORCAN’s 
creation in 1992, the Treasury Board 
stipulated that initially CORCAN would 
not pay for institutional structures and 
services, agricultural land and corporate 
services. A fee structure for these costs 
was to be developed and implemented in 
1995—96; this has not yet been done. If it 
were, we project that CORCAN’s costs 
would increase by millions of dollars, 
directly affecting its financial 
self-sustainability. 


Community Supervision 


Supervision of offenders in the 
community is crucial 


1.88 Supervision in the community is 
the final link in the offender reintegration 
process. It is the last point at which the 
system can directly influence or control 
offenders. It is also the point at which 
offenders are separated least from the 


| Level of Minimum Security Medium Security ] _ Maximum Security Exhibit 1.10 

Employment Needs* Institutions Institutions Institutions 

; F Employment Needs of Offenders 
Considerable 19 23) 46 Working for CORCAN 
— =< 2° a7 October 1998 
None 32 29 12 
Asset | 13 i | 3 

i | 

- Total 100% 100% 3, 100% 


* The Correctional Services rates needs into different levels of severity from “considerable need 


for improvement”, “some need for improvement”, 


“seen as an asset”. 


Sample: 
minimum — 410 
medium — 939 
maximum — 177 


Note: Excluded are 160 offenders whose employment needs were not readily available. 
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“no immediate need for improvement” to 


1,526 offenders working in institutions at various security levels: 


Source: Correctional 
Service Canada 
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Most offenders will 
complete their 
sentence in the 
community without 
re-offending. 


Exhibit 1.11 


CORCAN’s Financial Performance 


public and therefore present the greatest 
risk to society. 


1.89 Virtually all offenders currently 
in prison will some day be released into 
the community. Experience shows that 
most will complete their sentence in the 
community without re-offending. 
Nevertheless, when an offender in the 
community commits a violent offence, the 
lives of not only the victims but also their 
families can be shattered. 


1.90 In the Auditor General’s 1994 
Report, we indicated that it was crucial 
that supervision of offenders in the 
community be managed well. However, 
we noted that the Service management 
was not giving this area of corrections 
enough attention. We concluded that the 
senior management of the Service needed 
to provide more direction and guidance 
for the supervision of offenders in the 
community. 


More focus is placed on community 
supervision 


1.91 We recommended in 1994 that 
Correctional Service designate a senior 
official who would report directly to the 
Commissioner and be responsible for 
managing improvements in the practices 


WES 


for supervising offenders in the 
community. 


1.92 In November 1994, an advisor on 
community corrections was appointed. He 
chaired the National Community 
Corrections Council, whose objective was 
to enhance the contribution of community 
corrections to the safe reintegration of 
offenders in the community. The Council 
consists of Correctional Service senior 
managers in the community and 
representatives of non-government 
organizations. 


A permanent Director at 
headquarters subsequently replaced the 
position of advisor. Reporting to the 
Director General, Offender Programs and 
Reintegration, he is responsible for 
community corrections and chairs the 
National Community Corrections Council. 


1.94 In April 1997, most parole 
districts created the position of 
reintegration manager to improve the 
efficiency and effectiveness of the 
reintegration process. The role involves 
conducting special projects, monitoring 
caseloads and liaising with reintegration 
managers in the institutions. 


Required frequency of contact with 
offenders is still not maintained 


1.95 In 1994, we observed that 
face-to-face contact with the offender, as 


($ millions) ‘Actial Planned 
1993-94 1994-95 1995-96 1996-97 1997-98 | 1998-99 1999-00 
= + Le 

Revenue sy) 52 56 67 716 1S) 83 
Expenditures (54) (59) (58) (67) (79) (78) (84) 

| Profit or (Deficit) (2) (7) (2) 0 (3) (3) (1) 

| Unused portion of 

| $45 million limit 

| revolving fund 18 17 18 18 12 N/A* N/A 


* Figures not available 


Source: Actual figures from Public Accounts of Canada. Planned figures provided by CORCAN. 
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well as information obtained from the 
offender’s family, employer and program 
officers, were key components of offender 
supervision in the community. The 
average time available for direct 
supervision of offenders varied widely 
among district and local offices. We 
concluded that resources for supervision 
were not allocated appropriately. 


1.96 In February 1998, the 
Correctional Service found that there were 
still problems in achieving the frequency 
of contact with offenders that is required 
by policy. The percentage of offenders 
who were not contacted with the required 
frequency varied from a low of 11 percent 
to a high of 38 percent across the five 
regions. 


£97 In November 1998, we selected a 
random sample of 150 offenders under 
community supervision in five major 
urban centres. These were offenders 
considered by the Service to require the 
most frequent contact. The percentage of 
offenders in our sample who were not 
contacted with the required frequency 
ranged from 10 percent to 20 percent. 
These results are particularly significant 
given that our sample population was 
made up of offenders who require close 
supervision. 


1.98 One way to address this would be 
to ensure that parole officers in the 
community have reasonable caseloads. As 
a result of significant variances in parole 
office caseloads, Correctional Service 
recently initiated a study to establish a 
system for determining appropriate 
workload levels. The outcome of the study 
was a community workload formula that 
uses data from the automated offender 
information system. We applied this 
formula to June 1998 data on parole 
officer workloads and found that it would 
result in a more balanced distribution of 
caseloads than in 1994. The Service plans 
to approve and implement the formula 
during 1999. 
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1.99 Correctional Service Canada 
should ensure adherence to the 
standards for frequency of contact with 
offenders required by its policy, and 
should implement the new workload 
formula as planned. 


Correctional Service’s response: The 
Service agrees with this recommendation. 
CSC will continue to regularly audit the 
application of the supervision standards to 
determine if they are being met and will 
take firm and fair action regarding both 
compliance and non-compliance. The 
workload formula, a method for allocating 
resources based on activities required to 
supervise offenders, will be implemented 
nationally in June 1999. 


Approaches vary in managing offenders 
who need a high level of supervision 


1.100 In our 1994 Report we expressed 
concern that the Service was not 
identifying high-risk offenders 
consistently and accurately. During that 
year, the Service developed a course in 
risk assessment and provided training to 
all case management officers by the end 
of the fiscal year. Included in the course 
material was the National Parole Board’s 
definition of risk, which states that risk is 
determined on the basis of two primary 
considerations: 


e an assessment of the likelihood that 
the offender will re-offend; and 


e adetermination that, should the 
offender re-offend, there is no undue risk 
to the public, taking into consideration the 
nature and seriousness of the anticipated 
offence. 


1.101 We noted in 1994 that there were 
several different approaches used to 
manage these offenders in the community. 
In one area office, two people were 
assigned full-time to the high-risk 
caseload. Each offender was seen twice a 
week by a team of two parole officers — 
once in the office and once in the 
parolee’s home. In other area offices, 
however, mainly because of the workload, 
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offenders requiring close supervision 
might be seen only once a month, perhaps 
for as little as 15 minutes in the parole . 
office. 


1.102 In 1998, the Service approved a 
revised set of standards to specify how 
offenders should be supervised in the 
community. Several offices have 
developed related supervisory practices 
over and above these minimum standards, 
to deal with offenders who are particularly 
difficult to manage in the community. In 
one office, a team of two parole officers is 
assigned full-time to see each offender 
twice a week. In another office, difficult 
offenders are assigned only to the more 
experienced parole officers. In some 
offices there is a specialized unit for sex 
offenders. It has been a long-standing 
practice to manage these offenders using 
different approaches in different locations 
without any evaluation of their 
effectiveness. 


1.103 Correctional Service Canada 
should evaluate existing approaches to 
managing offenders who require a high 
level of supervision in the community in 
order to identify and implement the 
most cost-effective approaches under 
different circumstances. 


Correctional Service’s response: The 
workload formula will provide 
management with the information required 
to ensure more consistency in the types of 
supervision across CSC. At the same time, 
CSC will conduct evaluations of the 
results of such supervision practices 
regarding safe reintegration in order to 
determine their relative cost effectiveness 
and to make improvements. 


Managing Reintegration Activities 
to Achieve Desired Results 


1.104 Perhaps the greatest challenge 
any public organization faces is to manage 
its Operations and programs in a way that 
ensures desired results. Our 1997 Report, 
(Chapter 11 - Moving toward Managing 


for Results) outlined a framework for 
managing for results. We indicated that 
managing for results requires a supportive 
organizational environment (leadership, 
incentives, a capacity to learn and share 
experiences), agreement on expected 
results (outcome objectives, performance 
indicators and expectations) and the 
ability to measure and report results. We 
noted that making the change from 
managing inputs to managing for results 
takes time, usually four to five years. 


1.105 In 1996 (Chapter 30) we 
observed that there were persistent 
weaknesses in the management of 
reintegration activities across the Service. 
We concluded that there was a need for 
commonly accepted work standards across 
all regions, quality assurance procedures 
and performance information on results. 
We noted that the Service was having 
difficulty implementing basic changes, 
applying offender reintegration activities 
consistently across all regions and 
learning from its successes and failures. 


The continuity of offender risk 
assessment is improving 


1.106 To reintegrate offenders safely, 
the Service needs to consistently apply 
scientifically dependable risk assessment 
tools at each stage in the process — intake 
assessment, institutional case 
management, reports to the National 
Parole Board and supervision in the 
community. In 1994 we noted that there 
was no common set of practices and 
processes to identify and track changes in 
an offender’s risk and needs. At that time, 
the Service had already begun to address 
this concern. 


1.107 One of the key tools in the risk 
assessment process is the Custody Rating 
Scale, used to determine the appropriate 
institutional security level in which to 
place the offender. In some instances, the 
parole officer will use professional 
judgment to override the score indicated 
by the scale, and place the offender in a 
different security level. 
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1.108 Our 1994 audit found that the 
Custody Rating Scale was quantitative and 
objective; however, not all regions used it. 
We recommended that the Service 
monitor the rate of overrides and the 
appropriateness of decisions on security 
classification. By 1997, the Service had 
validated the Custody Rating Scale and 
made its use mandatory. 


1.109 In response to the Public 
Accounts Committee’s concern about the 
continuing high rate of overrides, 
Correctional Service adjusted the Custody 
Rating Scale in June 1998 so that 

34 percent of offenders would be placed in 
minimum-security institutions (previously 
15 percent). This reduced the national 
override rate from 25 percent to 

17 percent within a month. We could find 
no evidence that this change had a 
significant impact on the number of 
escapes by offenders between June and 
December 1998. 


1.110 Once the offender is placed in an 
institution, a security reclassification 
procedure is used periodically to 
determine whether his security level 
should be changed. The Commissioner 
assured the Public Accounts Committee 
that by the end of 1998 the Service would 
implement a new, more objective 
reclassification instrument. Since the 
beginning of 1996, the Service has 
devoted a considerable amount of work to 
developing a new, more quantitative 
instrument for reviewing security 
classifications. We can confirm that it has 
begun implementing the new 
reclassification instrument as promised. 


1.111 We also examined all risk 
assessment tools used by parole officers 
from the perspective of their scientific 
integrity, continuity and application as 
designed. We found that Correctional 
Service has adequately tested the 
scientific basis for the risk assessment 
instruments, especially the validity of 
their measurement of offender risk to 
reoffend. However, testing of their 
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reliability (that is, demonstrating that 
different parole officers will use the tool 
in a consistent and predictable way) is not 
as thorough as would be expected. 
Although some typical tests of reliability 
are available for these instruments, few or 
none have been conducted. It would be 
good practice for the Service to test the 


dependability of these tools. As promised the 


Quality controls are not applied Service has begun 


consistentty implementing a new 
1.112 In 1996 we reported that the reclassification 
mechanisms for quality control of release : 

recommendations were weak and the instrument. 


Service had no standard approach to 
quality control of reports that the National 
Parole Board used as a basis for making 
release decisions. 


1.113. In making these decisions, the 
National Parole Board places a great deal 
of emphasis on the quality of the parole 
officers’ analysis in reintegration reports 
submitted by the Correctional Service. 
The Service recognizes that the quality of 
its reports is a key factor in release 
decisions. The parole officer’s Case 
Management Manual states that the 
analysis is the most important stage in 
writing those reports. The Reintegration 
Task Force established by the Service to 
respond to our audits indicated similar 
concerns, and stated that the point at 
which the National Parole Board decides 
on an offender’s release is the point at 
which the report quality is most critical. 


1.114 The Case Management Manual 
outlines responsibilities for quality control 
of reports produced throughout the 
reintegration process. Reports going to the 
National Parole Board for release hearings 
are prepared by parole officers, who are 
responsible for ensuring that they are 
complete, properly formatted, on time and 
of appropriate quality. The Reintegration 
Task Force stated that institutional parole 
officers were the first point in the quality 
assurance process. 


1.115 In this audit, we found in 
reviewing reports to the Board on 
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offenders that quality controls were not 
applied consistently. Unit Managers 
(supervisor responsible for a team of 
correctional and parole officers) sign these 
reports and, in some institutions, the 
Co-ordinator of Case Management shares 
responsibility for their quality. However, 
the Reintegration Task Force found that 
Unit Managers currently cannot fulfil their 
role in assuring the quality of reports, 
because their background in case 
management is insufficient or they are not 


The quality of reports given enough direction, training or 


to the National Parole support. Their daily duties of managing 
the offender population generally take 


Board still requires precedence. We observed some use of 

improvement. “lead hand” senior case managers or 
specific quality control personnel, to help 
improve case management in general and 
report preparation in particular. The 
Reintegration Managers in some 
institutions have also helped with quality 
control. 


Offender report quality remains a 
concern 


1.116 In 1997 the Service conducted a 
National Review of Case Management, 
which assessed the quality of documents 
used by the National Parole Board in 
deciding on inmate releases. As detailed 
in Exhibit 1.12, the review found that of 


Exhibit 1.12 Quality Requirements Met 
(percentage) 
Quality of Reports to National 


Parole Board 783) 


Source: Final Report, National 


over 3,000 documents sampled across all 
regions, 48 percent to 78 percent complied 
with quality requirements. It concluded, 
“Supervision, direction and quality control 
appear to be absent or poorly delivered. 
..compliance and quality levels were 
generally below expectation.” One 
regional follow-up review that we 
examined, conducted in June 1998, 
indicated that the quality of case 
management was “effectively stagnant at 
1997 levels and requires immediate 
management action to improve 
performance”. 


1.117 = Weconducted our own test of 
report quality. We selected a random 
sample of all offender reports sent to the 
National Parole Board in 1998 for release 
decisions. The test was designed to 
determine the extent to which 
Correctional Service reports help Board 
members make informed release 
decisions. We examined the clarity, 
completeness and quality of analysis 
supporting recommendations on offender 
release. We also considered whether the 
quality of each offender report 
corresponded with the level of risk 
involved in the case. 


1.118 Our test of these reports 
examined whether they would enable the 
Board member to make a decision based 


Review of Case Management (1997) Atlantic Quebec Ontario Prairies Pacific National Total 
— Correctional Service Canada Correctional Service Regions 
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solely on the information in them (good 
reports); whether they had some gaps in 
information or analysis that would require 
additional review of the offender’s file 
(adequate reports); or whether they were 
of little value and the decision would 
require a full file review (poor reports). 
Our findings concurred with those of 
Correctional Service’s reviews. Overall, 
we found that 43 percent of the reports 
submitted to the Board for release 
decisions were “good”, 46 percent were 
“adequate”; and 11 percent were “poor”. 
Of reports on high-risk offenders, 

16 percent were of poor quality. 


1.119 Two major recurring concerns 
emerged across these reviews. The first 
was the number of reports sent to the 
Parole Board members that were 
incomplete and unclear. The second was 
the need for parole officers to improve 
their reporting of risk assessment and their 
analysis of the extent to which the 
offender has changed and is likely to be 
manageable in the community. 


1.120 Correctional Service Canada 
should address known deficiencies in the 
quality of its reports to the National 
Parole Board and ensure that quality 
control is exercised in accordance with 
approved policy. 


Correctional Service’s response: The 
Service will continue to place a high 
priority on ensuring responsibility and 
accountability for quality control is 
exercised by staff, supervisors and 
managers at all levels of the Service. 


The nature and extent of performance 
information has improved 


1.121 In 1996 we indicated that there 
was a lack of good performance 
information on case management 
activities. We said that Correctional 
Service managers at all levels (particularly 
at the institutional level) needed 
performance measurement systems to 
determine to what extent their activities 
were contributing to reintegration. We 
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encountered difficulty at that time finding 
any data on the results of the Service’s 
reintegration activities. 


1.122 The Service now has an increased 
capability to acquire, organize and 

analyze performance data in a more 

timely manner. Performance information 
that we gathered manually in 1996, on 


‘such things as the timeliness of acquiring 


documents and average time to achieve 
case management milestones, is now 
available from the Performance Assurance 
Branch at headquarters and from regional 
data systems. 


1.123. The Performance Assurance 
Branch prepares a Corporate Results Book 
for discussion at each executive 
committee meeting. The book outlines 
both national and regional performance in 
several aspects of the Correctional Service 
mission. It provides performance data and 
summary analysis on several key aspects 
of offender reintegration (offender 
admissions, releases, incidents, escapes, 
waivers and postponements). In addition, 
upon request the Research Branch 
undertakes statistical and trend analysis of 
data on the safe reintegration of offenders, 
covering such areas as offender population 
trends, offender reintegration without 
reoffence and offender potential for 
reintegration. 


1.124 The Service is also developing a 
Reintegration Tool Kit for local managers 
and parole officers to manage, plan and 
monitor the performance of reintegration 
activities on an ongoing basis. The first 
tool is Action Indicators, an instrument 
that monitors in each institution the 
ongoing status of offender reintegration 
documents received, offender assessments, 
steps completed in the case management 
process for each offender and status of 
offender reports against time standards. 
These tools are being designed by the 
supervisors and staff who will ultimately 
use them. Implementation is planned for 
early 1999. 
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The Service has now 
enhanced its capability 
to measure its 
performance. 


Efforts to implement a 
number of significant 
change initiatives have 
just begun and need to 
be sustained. 


1.125 The Service also uses national 
and regional reviews as another means of 
measuring local and regional compliance 
with approved policy and procedures. In 
February and March 1998, Correctional 
Service completed national reviews of the 
Intake Assessment units, Institutional 
Case Management and Community 
Supervision. In addition to assessing 
compliance with regulations, these 
reviews examined areas such as the 
quality and timeliness of reports and the 
quality of community supervision. These 
reviews and subsequent regional reviews 
have identified some performance 
deficiencies and requirements that the 
Operation Bypass initiative is addressing. 


1.126 The Service has now enhanced its 
capability to measure its performance at 
the national, regional and local levels. It 
recognizes that further refinement and 
acceptance of performance measures will 
be needed to improve managing for 
results. 


Lessons learned are being shared 


1.127 In 1994 and again in 1996 we 
observed that Correctional Service was 
not learning from its successes and sharing 
its best practices to make needed changes. 
Recently, however, the Service has made a 
concerted effort to share best practices 
among and within regions. 


1.128 Management and staff involved 
in offender reintegration activities, from 
intake to community supervision, have 
used such mechanisms as conferences, 
working groups and publications to share 
information on what is working and what 
is not. It is now an established practice 
that one region presents its current best 
practices at one of the two annual 
meetings of senior management. Meetings 
and workshops involving regional 
reintegration staff and National Parole 
Board officials are used to find ways to 
improve the quality of reports sent to the 
Board. 
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1.129 Correctional Service Canada has 
increasingly promoted and participated in 
the learning and sharing of best practices 
with other countries. In recent years, it has 
participated increasingly in international 
forums, international visits, training and 
assistance in correctional reform and 
technical assistance projects in developing 
countries. The Service has received 
several awards for its offender research, 
offender risk assessment tools and 
rehabilitation programs. 


Conclusion 


1.130 The findings of our previous 
audits led us to conclude at the time that 
there were systemic weaknesses in the 
Service’s management of its reintegration 
activities. We noted weaknesses 
throughout all aspects of the reintegration 
process. 


1.131 Since then, Correctional Service 
has made a concerted effort to respond to 
our observations through a wide range of 
initiatives. It has strengthened its national 
headquarters organization, which provides 
overall direction and co-ordination for 
changes in the reintegration process. It has 
begun to make substantive improvements 
in its intake assessment process, and has 
increased its efforts to prepare offenders 
for parole hearings in a timely manner. 
Program accreditation has had a positive 
impact on offender intervention programs. 
The Service now has the means to allocate 
workload in the community more 
effectively. 


1.132 Overall progress notwithstanding, 
improvement is still needed in some areas: 


e more timely acquisition of official 
documents for offender assessment; 


e more timely casework preparation to 
meet the offender’s first parole date; 


e aclear operational strategy for 
offender employment programs; 


e better-quality offender reintegration 
reports for the National Parole Board; and 
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¢ improved adherence to national 
standards for frequency of contact with 
offenders in the community. 


1.133 Efforts to implement a number of 
significant change initiatives have just 
begun and need to be sustained. As 
highlighted in the recommendations 
throughout this report, action is required 
to improve results in these critical areas. 
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1.134 At the end of our audit work in 
1996, we expressed concern about 
inconsistent practices and results across 
the five regions of the Service. We believe 
that the Service is now moving in the right 
direction. Its biggest challenge is to 
implement new initiatives effectively 
across all regions. In addressing this 
challenge, it must continue to move from 
managing inputs to managing for results. 
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About the Audit 


us 


Objectives 
The objectives of this audit were: 


e to determine the extent to which Correctional Service Canada (CSC) has acted on our earlier 
recommendations; and 


e to assess the extent to which the Service’s changes in the management of offender reintegration have 
contributed to sustainable improvements. 
Scope 


The focus of this audit was to re-audit key observations and recommendations made in our Report Chapter 30 
(1996) Reintegration of Offenders; Chapter 10 (1996) Rehabilitation Programs for Offenders and Chapter 18 
(1994) Supervision of Released Offenders. 


We focussed our examination on the major aspects of offenders’ reintegration, which included offender 
assessment and case management, offender programs, community supervision and the management of the 
reintegration activities. This included continuity of offender risk assessment, quality controls, performance 
information and sharing lessons learned. 


In response to a request by the Public Accounts Committee, the audit also included follow-up audit work on 
the Custody Rating Scale used to determine the initial security level of inmates’ institutional placement and 
the Security Reclassification instrument, which determines whether the security level of the inmate should be 
changed. 


This audit did not include female offenders or issues specific to Aboriginal offenders. 


Criteria 
We expected that: 


e CSC would have programs and policies in place that result in a sustainable management framework for 
offender reintegration, as demonstrated by: 


- leadership and management attention to address reintegration matters; 


— performance measurement systems that support operational decisions at all levels (national, regional, 
institutional); 


— the ability to assess, share and apply lessons learned to improve existing and future practices; and 
- a level of staff training and development to meet internal professional requirements. 


e CSC would perform effectively the critical functions that enable an offender (while incarcerated) to be 
ready for safe reintegration into the community at the earliest possible date. Performance of critical 
functions require: 


— proved and consistent risk assessment practices; 


- accurate and timely offender information from external agencies; 
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- accurate and timely assessment of offender criminogenic factors, community support and program 
requirements; 


— quality programs delivered on time; and 


— complete, timely and accurate reports to the National Parole Board for release decisions (Risk 
Assessment Profile). 


e CSC would supervise offenders in the community to facilitate safe and sustainable reintegration into 
society. Essential components of offender supervision in the community are: 


— continuity with institutional risk assessment practices; 
— availability of beds and the provision of required and relevant programming; 


— an appropriate and consistent level of parole officer contact with, supervision of and assistance to 
offenders; and 


— consistent methods used to minimize release revocations. 


Audit Team 


Assistant Auditor General: Maria Barrados 
Principal: Ronald Wolchuk 
Director: Gerald Rosinski 


Claude Brunette 
Robert Chen 

Sophie Chen 

Jayne Hinchliff-Milne 
William Johnson 
Chantal Michaud 

Dan Thompson 


For information, please contact Ronald Wolchuk. 
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Appendix 


Offender Population, Admission and Release Trends (1993-94 to 1997-98) 


Offender Population 


Offender Admissions 


Offender Releases 


Source: Correctional 
Service Canada 


Offender Location 1993-94 1994-95 1995-96 1996-97 1997-98 
= + 

Institution! 13,560 14,274 14,195 14,163 13,449 

59% 62% 63% 63% 61% 

Community? 9,405 8,688 8,415 8,246 8,744 

41% 38% 37% 37% 39% 

Total 22,965 22,962 22,610 22,409 22,193 


1 Institutional figures do not include escaped offenders and provincial offenders housed in federal 
institutions. (226 as of March 1998). 


N 


and offenders on parole who have been temporarily detained (667 at March 1998). 


Community figures do not include offenders deported upon release, and provincial offenders under 
federal supervision (787 at March 1998). Included are offenders unlawfully at large (707 at March 1998) 


1993-94 


Type of Admission 1993-94 1994-95 1995-96 1996-97 1997-98 
Warrant of Committal Syllity 4,783 4,401 4,569 4,501 
Revocation* without 2,174 2,434 2,148 2,346 2,389 
Offence 
Revocation* with 1,563 1,289 1,151 1,009 980 
Offence 
Total 8,854 8,506 7,700 7,924 7,870 
* Conditional release suspended and offender returned to federal prison. 
Type of Release 1993-94 1994-95 1995-96 1996-97 1997-98 
Day Parole 3,288 2,592 2,105 1,761 2,663 
(40%) (33%) (27%) (22%) (31%) 
Full Parole! 1,282 Oi 924 839 571 
(16%) (12%) (12%) (11%) (7%) 
Statutory Release 3,409 3,887 4,458 4,789 4,877 
(41%) (50%) (56%) (61%) (57%) 
Warrant Expiry 281 369 419 438 424 
(3%) (5%) (5%) (6%) (5%) 
Total 8,260 7,760 7,906 7,827 8,535 


i 


| 


1994-95 


1995-96 


1996-97 


1997-98 


| 1,262 


1,147 


908 


192 


1,315 


Source: National Parole Board 


These figures do not include offenders who change from day parole to full parole. 
These are shown below. 
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Chapter 2 


Revenue Canada 


Underground Economy Initiative 


The audit work reported in this chapter was conducted in accordance with the legislative 
mandate, policies and practices of the Office of the Auditor General. These policies and 
practices embrace the standards recommended by the Public Sector Accounting and 
Auditing Board (PSAAB) of the Canadian Institute of Chartered Accountants. The 
numbered paragraphs in bold face represent recommendations. 


Table of Contents 
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The underground economy 
At 4.5 percent of GDP, the underground economy warrants continuing government action 
Focus of the audit 


Observations and Recommendations 


Revenue Canada’s Strategy to Combat the Underground Economy 
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Implementation of the Underground Economy Initiative 
Results and impacts of the Underground Economy Initiative 


Legislative Opportunities to Deter Tax Cheats 


Conclusion 


About the Audit 
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25 The Underground Economy Initiative 

2.4 ' Tax Impact of $2.5 Billion Attributed to Activities to Combat the 


Underground Economy 
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Main Points 


2.1 Revenue Canada has reported that the tax impact (taxes resulting from enforcement actions) of its 
activities to address the underground economy was $2.5 billion over five years; however, this includes the results 
of both regular ongoing enforcement programs and the Underground Economy Initiative. The actual tax impact 
attributable to the detection of unreported income by the 1,000 staff allocated to Initiative audit activities is much 
less than the $500 million reported. 


2.2 It is difficult to assess the overall success of the Initiative in combatting the underground economy 
because the Department does not measure and report on the full range of Initiative activities and how they have 
changed taxpayer behaviour. 


23 Revenue Canada needs to assess the role that social marketing might play in making the public aware of 
the societal costs of unpaid taxes and in soliciting its support to combat the underground economy. Polls indicate 
that an alarming number of Canadians would be willing to participate in the underground economy. The 
Department also needs to strengthen the activities that promote voluntary compliance by businesses. 


2.4 The underground economy is a difficult and complex problem to solve and the size of the tax loss is 
significant. The problem requires continuous attention and sustained efforts from Revenue Canada and all 
Canadians. 


Background and other observations 


2.5 The underground economy results in tax evasion and represents an estimated annual loss of federal and 
provincial tax revenues of $12 billion. Tax evasion is not a victimless crime. It puts honest businesses at a 
competitive disadvantage and, in some cases, out of business. It also causes honest taxpayers to bear the tax load 
of those who cheat. If left unchecked, underground economy activity could lead to a loss of faith in the fairness of 
Canada’s tax system. 


2.6 In 1993, Revenue Canada announced a new initiative to combat the underground economy by allocating 
200 staff to its non-filers and non-registrants program and 1,000 staff to the audit of small businesses, where most 
of the underground economy activity exists. Thirty-five percent of the Department’s audit staff for small and 
medium-sized businesses are now involved in the Underground Economy Initiative audit activities. 


2.7 The Initiative as planned was, in our view, a balanced approach to combatting tax evasion in the 
underground economy. It included activities to promote voluntary compliance in small businesses such as 
community visits and consultations with industry associations. As well, it involved other federal departments, 
provincial and municipal governments and private sector organizations in sharing information to better deal with 
the underground economy. 


2.8 The Department can improve its targeting of audits for the detection and reassessment of unreported 
income. As well, legislative opportunities exist to strengthen existing incentives to deter participation in the 
underground economy. 


Revenue Canada has agreed to take action to address our recommendations. 
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Introduction 


The underground economy 


2.9 Definition. The “underground 
economy”, a term familiar to most 
Canadians, is commonly defined as 
activities that result in income that has 
been earned but not reported for tax 
purposes. The definition may include 
illegal activities, such as smuggling 
tobacco and liquor, drug trafficking and 
money laundering. 


2.10 In this chapter, we refer to the 
underground economy as legal 
transactions in goods and services that are 
“hidden”, resulting in the evasion of taxes. 
Examples of sectors where the 
underground economy is a concern 
include home repairs and renovations and 
auto repairs. Transactions in the 
underground economy often take place 
with cash because one party, or sometimes 
both parties, wants to make sure that no 
visible record exists. Participants in the 
underground economy cross all economic 
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sectors; they range from those who 
operate businesses on a full-time basis to 
“moonlighters”, and include those who do 
business with them. Exhibit 2.1 shows an 
example of a complex underground 
economy scheme. 


2.11 Factors often cited as 
contributing to the underground economy 
include higher tax rates, the growing 
complexity of the tax law, changes in 
public attitudes toward taxation and 
government in general, a low perceived 
risk of getting caught, opportunity (that is, 
no withholding of taxes at source), and a 
perception of minimal consequences. Tax 
commentators have also said that the 
“visibility” of the GST and the associated 
disillusionment that accompanied its 
introduction has fuelled the growth of the 
underground economy. 


2.12 Effects. The underground 
economy results in tax evasion, and tax 
evasion is not a victimless crime. It puts 
honest businesses at a competitive 
disadvantage and, in some cases, out of 
business. Exhibit 2.2 illustrates how the 


a tax return and related financial statements. 


| return was sent to audit. 


the work). 


A contractor had not filed his tax return. Revenue Canada issued a filing requirement and obtained 


A review of the tax return and financial statements showed that the contractor owned two 
corporations. The gross income of these two corporations was $3.5 million for three years. As the 
financial statements of these two corporations showed no salary expenses, Revenue Canada 
suspected that the contractor paid his employees in cash or “‘under the table”. The contractor’s 


Auditors found that the corporations were in reality selling invoices to other contractors (clients). 
On behalf of these clients and on their request, the corporations would send invoices to the 

clients’ customers for work done. Customers would pay the corporations. The corporations would 
keep 5 to 10 percent of the payments and would remit the difference to their clients (those who did 


| 
As the corporations did not keep proper books and records, auditors could not identify the clients 
| or, indirectly, the employees of these clients. Auditors also found that the contractor went 


bankrupt in August 1997, and also had in 1991. 


The $3.5 million of gross income reported by the corporations was in reality the salaries of the 


clients’ employees. No income tax had been withheld on these salaries by the corporations or the 
clients and remitted to Revenue Canada. Using an average tax rate of 20 percent, Revenue Canada 
estimated that this represented about $700,000 in lost federal taxes alone. The contractor never 
paid taxes on his commissions (from $175,000 to $350,000). The case illustrates how certain 
contractors used an old scheme — selling invoices — to conceal income. 


The underground 
economy results in tax 
evasion and 
represents an 
estimated annual loss 
of federal and 
provincial tax 
revenues of $12 billion. 


Exhibit 2.1 


A Complex Underground 
Economy Scheme - Example 


Revenue Canada identified the 
following scheme in reviewing 
a taxpayer's payments to 
various contractors. 
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New trends such as 
the recent growth in 
self-employment and 
the introduction of 
electronic commerce 
as a means of 
conducting business 
will create more 
opportunities to hide 
income from tax 
authorities. 


Exhibit 2.2 


How the Underground 
Economy Hurts Competition 


This exhibit illustrates how 
difficult it is for honest 
businesses to outbid businesses 
operating underground. 


underground economy hurts competition. 
Honest taxpayers lose business because 
customers may decide not to accept their 
bids or prices, which may be higher than 
those of vendors who fail to collect and 
remit the sales taxes, report their income 
and pay their fair share of income taxes. 


2.13 The underground economy 
causes honest taxpayers to bear the tax 
load of those who cheat and puts at risk 
essential government programs such as 
health and social services. If left 
unchecked, underground economy activity 
could lead to a loss of faith in the fairness 
of Canada’s tax system. 


At 4.5 percent of GDP, the underground 
economy warrants continuing 
government action 


2.14 Size. Estimates of the size of the 
Canadian underground economy contained 
in many studies over the last 17 years 

have ranged from 3 percent to over 

20 percent of gross domestic product 
(GDP). Our review of these studies 
indicates that differences in their 
objectives, definitions of the underground 
economy and measurement methodologies 


explain the wide variation in these 
estimates. Taking into account the 
differences, our review concluded that a 
reasonable range of estimates of the 
underground economy — defined in terms 
of the value of transactions in goods and 
services that are hidden and result in the 
evasion of taxes — was between 4.2 and 
4.5 percent of GDP in 1993. This range is 
in line with Statistics Canada’s estimate of 
4.2 percent of GDP for that year. At 

4.5 percent of GDP in 1997, the size of 
the underground economy would have 
amounted to $38 billion. This figure 
translates into a loss of income and 
commodity tax revenues of $12 billion for 
that year alone — $7 billion at the federal 
level and $5 billion at the provincial level. 


2.15 Trends. New trends such as the 
recent growth in self-employment and the 
introduction of electronic commerce as a 
means of conducting business will create 
more opportunities to hide income from 
tax authorities. 


2.16 The number of self-employed 
people grew by 38 percent from 1989 to 
1997. In 1997 they represented 18 percent 
of the employed population, compared 
with 14 percent in 1989. Revenue Canada 


By paying his workers “under the table”, an employer can reduce the cost for a worker employed 
legitimately by 40 to 50 percent. In general, this “cost reduction” arises from the non-payment of 
payroll taxes. The worker’s take-home pay may be increased by 30 to 40 percent when income taxes 
are not withheld at source and other payroll deductions are not made by the employer. 


Employer’s Employee’s 
costs take-home pay 
Regulated wage 
including benefits . $1400 $ 600 
Underground wage $ 800 $ 800 
Reduction or increase $ 600 $ 200 


This above-noted example illustrates the situation of a dry-wall worker working a 40-hour week. In 
the regulated employment situation, the employee is paid in accordance with agreed union rates for a 
dry-wall worker ($22.77 an hour in 1997) and employment benefits legislation. In the underground 
employment situation, the same dry-wall worker is paid at a rate of $20 an hour. This is a rate that 
Revenue Canada auditors have observed in the underground economy. The employee’s take-home 
pay is net of income tax and other payroll deductions. 
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has taken steps to make the self-employed 
more aware of their tax obligations. These 
include increasing enforcement actions 
and, in conjunction with the Department 
of Finance, conducting reviews to better 
understand the work patterns of the 
self-employed, their income levels, and 
the opportunities they have to avoid 
reporting income. 


2.17 Electronic commerce can hide 
the reporting of transactions for both 
income and commodity tax purposes. It 
raises practical difficulties of ensuring 
compliance, given that evidence about the 
identity and location of parties to a 
transaction may be difficult to obtain and 
assess. Existing record keeping and 
maintenance standards may not be 
sufficient to reflect electronic transactions. 
Revenue Canada may also have difficulty 
accessing encrypted evidence if it does not 
have a decryption key. Electronic 
commerce makes it very difficult to 
determine the source and nature of 
income. 


2.18 — Inits response to the 1998 report 
of the Minister’s Advisory Committee on 
Electronic Commerce, “Electronic 
Commerce and Canada’s Tax 
Administration”, Revenue Canada 
indicated that many of the issues raised by 
the Advisory Committee need further 
work. The Department intends to consult 
with the private sector in the development 
of new or revised tax compliance and 
administrative policies and procedures 
related to electronic commerce. 


Focus of the audit 


2.19 We examined the implementation 
of Revenue Canada’s Underground 
Economy Initiative to combat tax evasion 
in the underground economy. We also 
reviewed how the Department accounted 
for the Initiative’s results. Further details 
are found at the end of the chapter in the 
section About the Audit. 
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Observations and 
Recommendations 


Revenue Canada’s Strategy to 
Combat the Underground 
Economy 


The Underground Economy Initiative 
was announced in 1993 


2.20 In the early 1990s, there was a 
general perception that the underground 
economy was a growing problem and that 
the government had to do something more 
comprehensive to curtail it. In November 
1993, Revenue Canada announced a 
strategy to deal with the underground 
economy. The strategy focussed on 
unreported legal transactions in goods and 
services on which taxes had not been paid. 
It did not deal with unreported income 
from illegal transactions such as bribery, 
fraud and sales of drugs, which are 
addressed through other Revenue Canada 
programs. 


The Underground 
Economy Initiative as 
planned was, in our 
view, a balanced 
approach to 
combatting tax 
evasion in the 
underground 


PA Revenue Canada’s underground economy: 


economy strategy consists of seven 
initiatives, which are collectively referred 
to as the Underground Economy Initiative. 
Exhibit 2.3 illustrates the range of the 
activities planned as part of the Initiative. 
The objective of the Initiative is “to 
ensure integrity and fairness of the 
voluntary tax system by addressing 
underground economy activity and tax 
evasion to ensure a level playing field.” 
The Department expects that as a result of 
the Initiative, there will be increased 
voluntary compliance, changes in 
Canadians’ attitudes toward tax evasion, 
and increased compliance in targeted 
areas. 


Zod The announced Initiative was 
multi-dimensional. The Underground 
Economy Initiative as planned was, in our 
view, a balanced approach to combatting 
tax evasion in the underground economy. 
It was intended to increase the chance that 
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Exhibit 2.3 


unreported income would be detected, to 
enforce the payment of taxes from 
unreported income, to develop new 
activities to support taxpayers in meeting 
their tax responsibilities and to deter 
taxpayers from participating in the 
underground economy. Activities included 
encouraging voluntary compliance 
through community visits to business 
establishments, educating the public, 
partnerships with other governments and 
business organizations, enforcing 


The Underground Economy Initiative 


compliance by auditing in areas of high 
non-compliance and supporting 
compliance efforts with additional 


research. 


2.23 


Research undertaken by other tax 


administrations also points to the 
conclusion that a multi-dimensional 
approach is needed to influence the 
compliance behaviour of corporations and 
individual taxpayers, and to effectively 
combat the underground economy. 


REVENUE 
CANADA’S 
OBJECTIVE 


To ensure the integrity and fairness of the voluntary tax system by addressing underground economy activity 
and tax evasion to ensure a level playing field 


INITIATIVES 


ACTIVITIES 


Source: 


Work co-operatively 
with provinces, 
other government 
departments and 
key interest groups 


Vv 
+ joint audits 
* consultation 


+ exchange of 
information 


Strengthen 
programs to 


identify non-filers 
and 
non-registrants- 


Vv 


match 
databases 


increase 
enforcement 


Encourage 
voluntary 
compliance 


y 
news release 


public 
education 


community 
visits 


Enhance 
legislative 
effectiveness and 
audit techniques 


to 
| 
=> 


Compliance 
research 


y 


develop tools 
to identify 
high-risk 
sectors 


improve audit 
targeting 


analyze 
effectiveness 


Publicize tax 
evasion 
convictions 


v 

+ develop 
legislative 
proposals 


« refine audit 
techniques 


Adapted from Evaluation Study — Revenue Canada’s Underground Economy Strategy 


Focus on high 
non-compliance 
sectors 


V 


+ news releases 


v 


+ improve 
workload 
development 


+ audit high-risk 
sectors 
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Implementation of the Underground 
Economy Initiative 


2.24 Revenue Canada has 
strengthened its resources to deal with 
the underground economy in the last 
five years. The Department allocated 
1,200 staff to implement the Underground 
Economy Initiative. Of these, 

1,000 auditors are currently allocated to 
four targeted business sectors. The other 
200 staff were allocated to the non-filers 
and non-registrants program. 


2.25 In addition to the Initiative’s 
resources, a significant number of 
additional staff have been assigned to 
small business audits over the past 

three years. We found that these staff have 
been trained to audit small business 
taxpayers who have inadequate tax 
records. 


2.26 Revenue Canada has signed 
memoranda of understanding with all 
provincial governments to increase 
co-operation to combat the underground 
economy more effectively. It has also 
signed memoranda of understanding with 
Human Resources Development Canada 
to promote the exchange of information 
on Employment Insurance payments and 
income reported for tax purposes. 


2.27 The Department has consulted 
with private sector organizations and 
associations and has promoted voluntary 
compliance through visits to businesses. It 
has also been active in pursuing certain 
legislative changes such as the 
introduction of a new mandatory 
requirement for reporting income in the 
construction industry, and a new reporting 
system for all federal government services 
contracts. 


2.28 All Initiative resources were 
allocated to the enforcement area. The 
1,200 staff are essentially involved in 
enforcement activities such as auditing 
and detecting non-filers. The Department 
did not devote staff with specialized skills 
to implementing the voluntary compliance 
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activities of the Initiative. Enforcement 
staff have carried out some of these 
activities, while other activities have not 
been implemented. 


2.29 Proposed communication 
activities have not been carried out. The 
Underground Economy Initiative included 
activities aimed at making the public 
aware of the social inequities and costs 
arising from unpaid taxes and at 
generating support for Revenue Canada’s 
enforcement actions. Although a few 
specifically targeted communication 
projects have been carried out to date, 
Revenue Canada has not implemented 
general public awareness activities. The 
Department has not even conducted the 
necessary research to determine whether 
social marketing activities such as 
broad-based advertising could be a 
valuable tool to combat the underground 
economy. The Department has advised us 
that it has begun conducting such 


Surveys carried out by 
various organizations 
from 1994 to 1997 
show that an alarming 
number of Canadians 


research. 

would be willing to 
2.30 Documents in Revenue Canada’s participate in the 
possession indicate that surveys carried 
out by various organizations from 1994 to underground 
1997 show that an alarming number of economy. 


Canadians would be willing to participate 
in the underground economy. In December 
1994, a CTV Television Network poll 
indicated that 58 percent of Canadians 
would accept an offer to evade taxes when 
buying goods or services. In March 1997, 
a Gallup poll found that 73 percent of 
respondents said they would do so. While 
not necessarily indicative of a trend, these 
survey results suggest that Revenue 
Canada needs to involve the Canadian 
public more in combatting the 
underground economy. 


2.31 Revenue Canada should 
complete research to determine how to 
implement its social marketing activities 
to make taxpayers aware of the costs of 
the underground economy to society 
and legitimate business. 


> 
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In some communities, 
30 percent of the 
businesses contacted 
had either not filed a 
return or not 
registered for the GST. 


Department’s response: Revenue Canada 
agrees with the recommendation and will 
complete research to determine how to 
implement its social marketing activities. 
As reflected in the chapter, the 
Department is already conducting 
research to gain insights into how best to 
communicate about the underground 
economy. The Department's aim ts to: 


e determine how Canadians respond to 
concerns regarding the underground 
economy and its impacts on social 
programs; 


e determine whether a broad-based 
advertising campaign would be an 
effective way in which to raise general 
public awareness of the underground 
economy and of what the Department 
has been doing to deal with it, and, if 
so, what messages and media would 
have the most impact; and 


e improve and expand its existing 
communications efforts targeted at 
specific industry sectors and related 
consumers. 


Social marketing will be a long-term 
initiative. In addition to extensive 
research, social marketing entails 
obtaining the support and active 
involvement of a number of partners. 
These partners include other federal 
government departments and the 
provinces. Also, we will continue our 
co-operative communications activities 
with private sector associations. 


The Department has met with a number of 
stakeholders to explore interests and 
commitment. We are also currently 
conducting public opinion research in the 
home renovation sector, to be completed 
by 31 March 1999, with the view of 
developing a social marketing plan and 
other communication-related strategies for 
that sector, as well as for the general 
public. 


2S. Activities undertaken to 
promote voluntary compliance by 
businesses have declined. The 


bo 


i) 


Underground Economy Initiative included 
activities to increase voluntary 
compliance by business taxpayers. Two of 
these activities were visits to business 
establishments in selected communities 
and increased consultations with business 
associations. 


2.33 Revenue Canada has undertaken 
visits to business establishments to remind 
them of their tax responsibilities and to 
note taxpayer names and business 
numbers for future follow-up action. 
During these visits, staff provide taxpayers 
with a package of information to help 
them voluntarily comply. Revenue 
Canada’s quarterly reports on the 
Initiative’s activities show that the number 
of visits to businesses has declined over 
the past two years. In 1995—96 and 
1996-97, over 120 community visits — or 
an average of 60 per year — were made. 
In 1997-98, Underground Economy 
Initiative auditors made only 

37 community visits. Consultations were 
initiated with over 400 business 
associations, but quarterly reports indicate 
that few follow-up meetings have been 
held over the past year. 


2.34 The recent decline of efforts in 
community visits and consultations is 
worrisome, as it diminishes the role that 
the activity referred to as “encourage 
voluntary compliance” can play in 
combatting the underground economy. 


pa) It is also worrisome that the 
Department’s response to the information 
obtained during the community visits has 
often consisted of enforcement actions. 
These community visits have indicated 
high rates of non-compliance. In some 
communities, for example, 30 percent of 
the businesses contacted had either not 
filed a return or not registered for the 
GST. The Department’s response to this 
situation was to refer the files to the 
non-filers program or to the audit 
function. Other follow-up activities to 
assist the taxpayers in understanding and 
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meeting their tax reporting responsibilities 
could be used. 


2.36 We noted, for example, that to 
help reduce non-compliance in the 
transition to the Harmonized Sales Tax, a 
Tax Services Office had undertaken an 
innovative practice of extending 
assistance to taxpayers. Using its Client 
Services staff, it started an outreach 
program that involves meeting with 
taxpayers individually to better explain 
their tax obligations and to promote 
voluntary compliance with the new 
Harmonized Sales Tax. Although not an 
Underground Economy Initiative activity, 
the Department considers this program 
successful in attacking the underground 
economy. 


DST Revenue Canada should 
strengthen its Underground Economy 
Initiative activities to promote 
voluntary compliance by businesses. 


Department’s response: Revenue Canada 
agrees with the recommendation and has 
always made activities to promote 
voluntary compliance a key part of the 
Department’s balanced approach to 
addressing the underground economy. As 
noted in the chapter, Revenue Canada 
undertakes numerous activities to promote 
voluntary compliance. Emphasis will be 
placed on increasing the number of 
community visits and on maintaining 
relationships that have been established 
with private sector associations. Also, the 
social marketing research being 
undertaken by the Department will 
provide useful information on the types of 
activities that are most likely to be 
effective in encouraging voluntary 
compliance. 


In particular, Revenue Canada considers 
its community visits to be an excellent 
opportunity to assist taxpayers in 
understanding and meeting their tax 
reporting responsibilities. In addition to 
leaving taxpayers with a package of 
information to help them comply 
voluntarily, staff offer to answer questions 
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taxpayers might have and to provide 
assistance on-the-spot or shortly 
afterward. Appropriate enforcement action 
is taken only if a taxpayer has still not 
complied after information and assistance 
have been provided. 


2.38 External sources of information 
can improve detection of unreported 
income. The underground economy is 
generally characterized by a large number 
of small transactions that collectively 
amount to several billion dollars of 
unreported income. In this context, the 
efficient detection and assessment of 
relatively small amounts of unreported 
income pose enormous challenges to 
Revenue Canada. 


2.39 The use of external sources of 
information complements Revenue 
Canada’s internal information and can 
improve the identification of non-filers 
and the selection of audit files. These 
sources yield additional information about 
both the income paid to taxpayers and the 
wealth of taxpayers identified as possibly 
not having fulfilled their tax 
responsibilities. Such information 
contributes to detecting the worst cases of 
unreported income and providing more 
certainty before an enforcement action is 
undertaken. 


2.40 Over the last five years, the 
Department has obtained more than 

70 databases from other federal 
departments, provincial and municipal 
governments and private sector sources 
and matched them against its own 
databases. For example, one Tax Services 
Office (TSO) obtained a private sector 
database of payments made to contractors, 
and preliminary results indicate that 
substantial unreported income may exist 
in a segment of the resources sector. 


Another TSO received a database of 
payments made to city construction 
contractors. When matched against the 
contractors’ filed returns, a sample of 
payments indicated that 40 percent had 
not reported these payments. At the time 


Over the last 


five years, Revenue 
Canada has obtained 


more than 70 


databases from other 
federal departments, 


provincial and 
municipal 
governments and 


private sector sources 


and matched them 
against its own 
databases. 
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of our audit, the Department was 
finalizing its enforcement response. 


2.41 We found that access to 
provincial information varied from region 
to region and among TSOs. For example, 
only one provincial government provided 
a database on workers compensation; two 
provided information on property 
assessments and one on land registries. 
Most provided information on motor 
vehicle registration. Our examination also 
showed that one particular Tax Services 
Office had requested 22 provincial 
databases, but after 18 months the 
majority still had not been received. 


2.42 The Department has signed 
memoranda of understanding with all 
provinces to increase co-operation in 
dealing with the underground economy. 
Most provinces have identified a 
representative to work with Revenue 
Canada to facilitate the data exchange. 
The provinces benefit from Revenue 
Canada’s activities to counter the 
underground economy because under 
current tax collection agreements, they 
receive the provincial share of any tax 
reassessment flowing from Revenue 
Canada’s audits. 


2.43 We recognize that governments 
are obliged to protect the privacy of 
information provided to them, and a 
process is needed to allow for the proper 
exchange of information between federal 
departments and other levels of 
government. However, it is also important 
for Revenue Canada to have efficient 
access to data that can be used to detect 
unreported income, especially income 
from government payments. 


2.44 Revenue Canada should 
collaborate with other levels of 
government and other partners to 
access the data that can be used to 
detect unreported income. 


Department’s response: Revenue Canada 
agrees with the recommendation and will 
continue to collaborate with the other 


levels of government and other partners to 
access the required data. As discussed in 
the chapter, Revenue Canada makes 
extensive use of external sources of 
information to complement internal 
information, to identify non-filers and to 
assist in selecting files for audit. Matching 
of Revenue Canada databases with 
external databases has been and continues 
to be a key tool used by the Underground 
Economy Initiative in identifying and 
auditing cases of unreported income. As 
paragraph 2.42 indicates, Revenue 
Canada has agreements with all provinces 
to increase co-operation in dealing with 
the underground economy. The agreements 
facilitate the exchange of information, as 
well as other co-operative activities. The 
existence of these arrangements 
demonstrates the importance that the 
Department places on the use of external 
databases. 


Revenue Canada recognizes the 
compliance benefits that accrue from 
increased sources of data. As the 
Department continues to work in closer 
collaboration with its provincial 
colleagues, and as it moves to proposed 
Agency status and explores new business 
opportunities with them, information 
sources of benefit to the Department's 
programs will be explored. 


At the same time, Revenue Canada is 
committed to ensuring that the 
confidentiality provisions of the legislation 
administered are fully respected, and that 
the Treasury Board Guidelines regarding 
data matching are observed. Furthermore, 
the compliance benefits of data matching 
must be carefully balanced with 
Canadians’ regard for privacy. 
Consultation with the Office of the 
Privacy Commissioner is undertaken on 
an ongoing basis, as required. 


2.45 More research is needed to 
support Underground Economy 
Initiative activities. Auditors need a good 
knowledge of the sectors they are auditing 
if they are to be effective at detecting and 
assessing unreported income and 
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combatting local underground economy 
activity. They need to understand, for 
example, the nature of the operations in 
those sectors. They also need to be able to 
assess the risks of non-compliance for a 
given sector and to develop an appropriate 
compliance approach. 


2.46 Revenue Canada committed itself 
to undertaking research aimed at gathering 
appropriate information to support the 
enforcement activities included in its 
Underground Economy Initiative. 
Departmental documents refer to the 
importance of developing sector profiles 
to assess the risk of unreported income in 
economic sectors and to understand the 
business operations of these sectors. 
Accordingly, we inquired into whether the 
Department had developed profiles for the 
four sectors that it had decided to target in 
carrying out its Initiative: construction and 
renovation; used car sales and auto 
repairs; hospitality; and jewelry. We found 
that the Department had not completed the 
profiles for all the sectors when it decided 
to focus on them. Two sector profiles are 
still in draft form. 


2.47 Revenue Canada should 
complete the development of its sector 
profiles in order to better audit 
underground economy issues. 


Department’s response: As noted in the 
chapter, departmental documents refer to 
the importance of developing sector 
profiles to assess the risk of unreported 
income in economic sectors and to 
understand the business operations of 
these sectors. Revenue Canada agrees 
with the recommendation to complete the 
development of these sector profiles and is 
currently in the process of doing so. 


2.48 Taxpayers with a high risk of 
unreported income exist in all sectors. 
In narrowing the focus of Underground 
Economy Initiative audits to four sectors 
(construction, jewelry, hospitality and 
automotive), the Department wished to 
concentrate its enforcement resources 
where it believed they would have the 
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largest and most visible impact. 
Sixty-three percent of Underground 
Economy Initiative audits completed in 
1997-98 in these four sectors were related 
to the construction sector. 


2.49 Taxpayers with high risk of 
unreported income exist in all economic 
sectors. However, audits conducted by 
regular business auditors in these sectors 
may not focus as strongly as Underground 
Economy Initiative audits on identifying 
unreported income. It may be time to 
reconsider the distinction between small 
business audit and Initiative audit and 
expand the focus on unreported income to 
the highest-risk taxpayers in all sectors. 
The necessary training and management 
guidelines for team leaders need to be in 
place to support this broader approach to 
combatting tax evasion. 


2.50 Revenue Canada should 
reconsider its audit focus for the 
Underground Economy Initiative to 
include taxpayers in all sectors with a 
high risk of unreported income. 


It may be time to 
reconsider the 
distinction between 
small business audit 
and Underground 
Economy Initiative 
audit and expand the 
focus on unreported 
income to the 
highest-risk taxpayers 


Department’s response: Revenue Canada in all sectors. 


agrees to consider expanding its audit 


focus for the Underground Economy 


Initiative to include taxpayers in all 
sectors with a high risk of unreported 
income. Any expansion would have to take 
into consideration the resource 
implications and be balanced against the 
Department's ability to maintain a large 
and visible impact in the selected sectors. 


Results and impacts of the Underground 
Economy Initiative 


2.51 Five years have passed since the 
Underground Economy Initiative was 
announced. We expected the Department 
to measure and report clearly the tax 
impact and other results of the Initiative’s 
activities. This information is important to 
make decisions about the future of the 
Initiative — whether it should be 
discontinued, maintained, expanded or 
integrated in ongoing departmental 
operations. The information is also 
important to determine what strategies are 
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Revenue Canada has 
reported that the tax 
impact of its activities 
to address the 
underground economy 
was $2.5 billion over 
five years; however, 
this includes the 
results of both regular 
ongoing enforcement 
programs and the 
Underground 
Economy Initiative. 


Exhibit 2.4 


the most effective in combatting the 
underground economy. 


2.92 It is difficult to assess the 
success of the Underground Economy 
Initiative enforcement activities in 
detecting unreported income. The 
Department has reported that its activities 
to combat the underground economy 
resulted in a $2.5 billion tax impact (taxes 
resulting from enforcement actions) for 
the period November 1993 to 31 March 
1998. However, the $2.5 billion includes 
both the results of regular ongoing 
enforcement programs and the results of 
the Initiative. 


2.53 We examined the $2.5 billion for 
results attributable to the Underground 
Economy Initiative (see Exhibit 2.4). 


2.54 The $2.5 billion includes an 
amount of $1.6 billion that represents the 
total tax impact of the non-filers and 
non-registrants program for that period. 
This program is a regular enforcement 
program that has existed for many years. 
The Underground Economy Initiative only 
intensified the use of this program by 


Tax Impact of $2.5 Billion Attributed to Activities 
to Combat the Underground Economy 


(November 1993-31 March 1998) 


Non-filers and non-registrants program 
(regular and Initiative) $ 1.6 
Underground Economy Initiative audits 
Revenue Canada 
Income Tax $ 3 
GST £2, 
5) 
Ministere du Revenu du Québec 
GST ally 6 
Non-Initiative audits of self-employed taxpayers iP 
Non-Initiative special projects in 1994-95 sil 
Total ie $ 2.5 


Source: Revenue Canada 
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adding 200 staff to its 550 pre-existing 
staff. The Department has never reported 
the incremental tax impact of the 

200 additional staff. During this time, the 
Department also reported that the number 
of returns obtained as a result of the 
non-filers and non-registrants program 
dropped from 509,000 in 1993-94 to 
441,000 in 1997-98. 


255 Of the reported $2.5 billion tax 
impact, about $600 million accounts for 
audits that the Department has recorded as 
Initiative audits. Initiative income tax and 
GST audits account for about 

$300 million each. 


2.56 Of the $300 million GST tax 
impact, about $100 million resulted from 
GST audits done by ministére du Revenu 
du Québec officials. The resources 
allocated to the Initiative were not used to 
carry out these audits. 


2.57 Of the $300 million resulting 
from income tax audits, we were 
interested in determining how well the 
Underground Economy Initiative audits 
had detected and reassessed unreported 
income. Revenue Canada’s systems do not 
record separately the amount of additional 
gross income that its auditors have 
identified and reassessed. This makes it 
difficult for the Department to account for 
the impact of the Initiative’s audit 
activities in addressing unreported 
income. 


2.58 We reviewed over 90 Initiative 
income tax audit files from four Tax 
Services Offices to identify the unreported 
income detected by these audits. Over 

40 percent of these files did not involve 
unreported income; rather they included 
reassessments due to technical matters 
such as incorrect capital cost allowance 
claims, disallowance of tax shelter losses 
and other losses, and income arising from 
shareholder loans. Although the tax 
impact of these types of reassessments 
was attributed to the Initiative, it 1s 
unrelated to underground economy 
activity. 
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2.59 The results of our file review led 
us to conclude that the tax impact 
attributable to unreported income found in 
these Initiative income tax audits was 
significantly less than the reported 

$300 million for that period. This suggests 
that the targeting and selection of 
Initiative income tax audit files need to be 
improved to increase the effectiveness of 
the Department’s efforts in combatting the 
underground economy. 


2.60 In total, the tax impact 
attributable to 1,000 staff assigned to 
Initiative income tax and GST audits over 
the last five years is about $500 million. 
In our view, the tax impact attributable to 
unreported gross income as a result of 
Initiative audits is much less than the 
$500 million reported. 


2.61 Collection of the $2.5 billion 
impact is uncertain. As we have pointed 
out in previous audits, Revenue Canada’s 
systems do not track how much of the 
reassessed taxes the Department actually 
collects. This is important because under 
tax collection agreements, provinces are 
paid for taxes assessed on their behalf, 
even if they are not collected. 


2.62 Revenue Canada should record 
and report the additional gross income 
identified by its Underground Economy 
Initiative and non-Initiative 
enforcement activities, the additional 
tax due on this unreported income, and 
how much of the reassessed additional 
taxes the Department actually collects. 


Department’s response: Outstanding 
accounts receivable result from a number 
of tax assessments including income tax, 
GST/HST (Harmonized Sales Tax) and 
source deductions. Such assessments can 
relate to unreported gross income, as well 
as to other adjustments. The net amount 
owing can be affected by various items 
such as the availability of loss carry 
forwards and carry backs and the 
assessment of penalties and interest. 
Furthermore, payment of taxes occurs in 
many ways including offsets and 
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installments. Associating the source of the 
tax, or the specific type of adjustment 
relating to that tax, with a payment in 
respect to an amount owing would require 
a number of assumptions, as a direct 
linkage would not appear to be feasible. 


Although Revenue Canada’s systems 
currently record the aggregate amount of 
adjustments made to income and expenses, 
the systems do not separately record the 
amount of additional gross income. 
Recognizing that modifications to systems 
can be costly and must be balanced 
against other departmental priorities, the 
Department will explore options to record 
the additional gross income identified by 
its enforcement activities. 


The tax impact 
attributable to 
unreported gross 
income as a result of 
Underground 
Economy 

Initiative audits is 
much less than the 
$500 million reported. 


Notwithstanding the complexity of 
matching as discussed above, Revenue 
Canada will also explore options to 
improve the reporting of collection activity 
in terms of how much of the additional 
taxes reassessed from underground 
economy activities is actually collected. 


2.63 Revenue Canada should 
improve the targeting and selection of 
income tax files to increase the 
effectiveness of its Underground 
Economy Initiative audit activities in 
identifying unreported income. 


Department’s response: As noted in the 
chapter, the Initiative involves a balanced 
approach with one of its main objectives 
being to detect unreported income. 


The Department has been making 
significant improvements to its risk 
assessment systems using external data 
sources to detect unreported income and 
estimate taxes at risk. Income tax files that 
these risk assessment systems identify as 
being at high risk for unreported income 
will be used as a basis for file selection 


for Initiative audits. 


In addition, as indicated in paragraph 
2.71, the Contract Payment Reporting 
System for the construction industry, 
which was implemented on a mandatory 
basis beginning January 1999, will 
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Appropriate indicators 
of performance for the 
Underground 
Economy Initiative 
need to be identified 
and publicly reported. 


provide data that will assist the 
Department in finding unreported income 
by matching reported payments to 
subcontractors against income reported on 
their tax returns. 


Revenue Canada will continue to improve 
the targeting and selection of income tax 


files to increase the effectiveness of its 


Initiative audit activities. 


2.64 Performance indicators are not 
in place. As indicated in paragraph 2.21, 
the objective of the Underground 
Economy Initiative is “to ensure integrity 
and fairness of the voluntary tax system 
by addressing underground economy 
activity and tax evasion to ensure a level 
playing field.” The expected effects of the 
Initiative are increased voluntary 
compliance, changes in Canadian’s 
attitudes toward tax evasion, and 
increased compliance in targeted sectors. 


2.65 To date, the principal 
performance measurement that Revenue 
Canada uses to assess the results of its 
actions against the underground economy 
is the tax impact. 


2.66 The Underground Economy 
Initiative includes various important 
activities that may not result in an 
immediate tax impact, but that are 
intended to bring about behavioural 
change, leading to longer-term 
compliance of taxpayers. Because 
behavioural change is difficult to measure, 
a wide range of performance indicators is 
needed to measure and report on the 
performance of the Initiative activities and 
on their long-term effects on compliance. 


2.67 The Department has considered 
several indicators that could be used to 
measure and report the results of the broad 
range of Initiative activities. These 
include information on the number of 
non-filers identified and the tax found to 
be owing following a community visit; the 
number of voluntary disclosures or 
informant leads following a community 
visit; the number of non-filers who 
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continue to file after being served with a 
requirement to file; the number of 
taxpayers who, following an audit, 
continue to report incomes at the level 
identified by the audit; the number of 
requirements issued to keep books and 
records; the number of taxpayers who, 
after being required to keep proper books 
and records, continue to do so; the number 
of penalties for negligence issued 
following an audit; and the number of 
net-worth audits completed. However, the 
quarterly reports of the Initiative and the 
Department’s annual Performance Report 
do not contain performance information 
on the full range of Initiative activities. 


2.68 We noted that the Department 
was developing certain tools to better 
enable it to assess the performance of all 
enforcement activities, including the 
Underground Economy Initiative. One of 
these tools is a new core audit program. 
Under this program, a sample of randomly 
selected returns will be audited to estimate 
the extent of non-compliance in a 
particular taxpayer population. One of the 
Department’s objectives for the program is 
to obtain statistically valid rates of 
non-compliance by industry sector, and an 
insight into the relative importance of 
major sectoral issues such as unreported 
income and sales, overstated expenses or 
technical issues. A pilot project is 
expected to be conducted in 1999-2000. 


2.69 Appropriate indicators of 
performance for the Underground 
Economy Initiative need to be identified 
and publicly reported. Without them, 
Revenue Canada cannot account for either 
the performance of the various Initiative 
activities — individually or collectively 
— or the extent to which the activities 
have changed taxpayer behaviour. 


2.70 Revenue Canada should 
develop indicators to measure and 
report the results of the full range of 
Underground Economy Initiative 
activities. 
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Department’s response: As acknowledged 
in the chapter, Revenue Canada has 
considered several indicators that could 
be used to measure and report the results 
of the broad range of Initiative activities. 
In addition, although behavioural change 
is difficult to measure, a wide range of 
performance indicators is needed to 
measure and report on the performance of 
the Initiative activities and on their 
long-term effects on compliance. 


As indicated in paragraph 2.68, Revenue 
Canada is in the process of developing a 
core audit program to better enable it to 
assess the performance of all enforcement 
activities, including the Underground 
Economy Initiative. The core audit 
program started in January 1999. 
Guidelines, including a listing of the 
audits that must be completed as part of 
this program by the end of December 
1999, have already been provided to field 


staff. 


In addition, in its compliance performance 
reported entitled Compliance: From 
Vision to Strategy, the Department has 
published performance indicators that 
relate in part to the underground economy. 
These indicators indicate the rate of filing 
of individual tax returns and the level of 
self-employment income reported for tax 
purposes. In both cases, these indicators 
demonstrate some improvement in 
compliance. 


Nevertheless, Revenue Canada agrees 
with the recommendation and will 
continue to implement additional 
indicators for 1999-2000. 


Legislative Opportunities to Deter 
Tax Cheats 


pg fh Reporting requirements. 
Beginning | January 1999, construction 
contractors will be required both to record 
all payments they make to subcontractors 
who provide construction services and to 
report those payments to the Department. 
The reporting system has been operating 
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voluntarily since 1996 and has now 
become mandatory. These data will assist 
Revenue Canada to find unreported 
income by matching reported payments to 
subcontractors against their tax returns. 
The Department plans to look at 
expanding similar reporting requirements 
to other sectors that provide home 
renovation services. 


Dz Legislation has also been 
introduced to require that federal 
government departments and Crown 
corporations issue information slips for 
services they purchase from suppliers. 
Departments will begin issuing 
information slips in 1999 and Crown 
corporations will begin the next year. 


Revenue Canada could 
pursue legislative 
opportunities to deter 
tax cheats. 


2.13 Requesting a refiling. Revenue 
Canada could pursue legislative authority 
to undertake another form of enforcement 
action. For instance, the State of 
Massachusetts can compel a taxpayer who 
has filed an insufficient or incorrect return 
to determine the correct tax lability and 
file a proper return. The law allows a 
penalty of up to double the amount of tax 
owing if it is subsequently found that the 
taxpayer failed to file a proper return. This 
provision comes into play once there is 
probable cause. Some basis of fact must 
exist that gives cause to believe that a 
return is insufficient or incorrect. 


2.74 Monetary penalties. In each of 
the last three years, Revenue Canada 
issued about 5,000 requirements to file an 
income tax return and charged more than 
2,000 taxpayers for failing to file. The 
prosecutions that follow take place in the 
court system and include a minimum 
$1,000 fine. Revenue Canada could 
consider prescribing an administrative 
monetary penalty, both for failing to file 
an income tax return when a requirement 
has been issued and for failing to keep 
adequate books and records. The 
Department could also impose more 
onerous record-keeping requirements on 
taxpayers with a serious history of 
underreporting. 
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Revenue Canada has 
so far focussed its 
efforts on enforcement 
activities and has not 
fully implemented 
other key activities 
included in its 
Underground 
Economy Initiative. 


Zale Reporting cash transactions. 
Several countries now have legislation 
requiring the reporting of cash 
transactions over a certain amount. 
Canadian legislation currently requires 
recording of these transactions by banks, 
but not centralized reporting to an agency 
mandated to follow up on suspicious 
transactions. The reporting is helpful in 
discovering money-laundering activities 
and in tracking cash sales, which may 
result in unreported income for tax 
purposes. 


Conclusion 


2.76 Revenue Canada has always had 
programs to detect and deter the 
non-reporting of income. In 1993, in 
response to a perceived growth in the 
underground economy, the Department 
announced a new initiative. As planned, 
Revenue Canada’s Underground Economy 
Initiative was to include 1,200 staff in a 
multi-dimensional and balanced approach 
to combat the underground economy. 


2.77 In implementing the Initiative 
over the past five years, Revenue Canada 
has strengthened its resources to detect 
and audit unreported income by allocating 
more staff to its non-filers and 
non-registrants program and to the audit 
of small businesses, where most of the 
underground economy activity exists. 
Thirty-five percent of its audit staff for 
small and medium-sized businesses are 
now involved in Initiative audit activities. 


2.78 It is difficult to assess the success 
of the Initiative enforcement activities in 
detecting unreported income. The 
Department includes both the results of 
regular ongoing enforcement programs 
and Initiative activities in the reported 
results of a $2.5 billion tax impact. In 


addition, it does not separately record 
unreported income and the related 
additional tax identified through audit. 
However, in our view, the tax impact 
attributable to the detection of unreported 
income by the 1,000 staff allocated to 
Initiative audit over the five years is much 
less than the $500 million reported. 


219 The Department has so far 
focussed its efforts on enforcement 
activities and has not fully implemented 
other key activities included in its 
Initiative. The underground economy 
involves both businesses and their 
customers. Revenue Canada needs to 
complete research to determine how to 
implement its social marketing strategy to 
make the public aware of the societal 
costs of unpaid taxes and to solicit its 
support in combatting the underground 
economy. The Department also needs to 
strengthen the activities that promote 
voluntary compliance by businesses and 
improve its targeting of audits to detect 
and reassess unreported income. As well, 
there are legislative opportunities to 
strengthen existing deterrents to 
participation in the underground economy. 


2.80 The Department has not reported 
appropriate performance indicators and 
measured the results of the full range of 
Initiative activities intended to encourage 
taxpayers to report all their income. 


2.81 Through the Underground 
Economy Initiative, Revenue Canada has 
recovered some additional taxes and sent a 
message to tax cheats in some sectors that 
it is determined to deal with the problem. 
However, the underground economy is a 
difficult and complex problem to solve 
and the size of the tax loss is significant. 
The problem requires continuous attention 
and sustained efforts from Revenue 
Canada and all Canadians. 
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About the Audit 


us 


Objective and Scope 


The objective of the audit was to determine how well Revenue Canada has implemented its Underground 
Economy Initiative and reported the results. 


The audit took place at Revenue Canada headquarters in Ottawa and at various departmental offices 
throughout the country. 


Criteria 

To combat the underground economy, we would expect the Underground Economy Initiative to have in place: 
e adequate information on the size of the underground economy and the areas of high non-compliance; 
e appropriate tools and techniques to detect, reassess and deter underreporting of income by taxpayers; 
e effective activities to promote taxpayers’ awareness of their income reporting obligations; 
@ appropriate measurement and reporting of the impacts of the Initiative; and 


e effective co-operation with other jurisdictions and other government departments. 


Audit Team 


Assistant Auditor General: Shahid Minto 
Principal: Barry Elkin 
Directors: Richard Gaudreau, Brenda Siegel and Abid Raza 


For information, please contact Barry Elkin. 
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Chapter 3 


Statistics Canada 


Managing the Quality of Statistics 


The audit work reported in this chapter was conducted in accordance with the legislative 
mandate, policies and practices of the Office of the Auditor General. These policies and 
practices embrace the standards recommended by the Public Sector Accounting and 
Auditing Board (PSAAB) of the Canadian Institute of Chartered Accountants. The 
numbered paragraphs in bold face represent recommendations. 
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Statistics Canada 


Managing the Quality of Statistics 


Main Points 


3.1 Statistics Canada is committed to producing statistics of high quality. It has put in place a wide range of 
systems and practices to build quality into its statistical programs and to maintain an environment that encourages 
a concern for quality throughout the organization. However, the quality of the statistics it produces needs to be 
better documented and reported both within and outside the Agency. The Agency needs to integrate its many 
quality-related systems and practices better and adopt a more disciplined approach to documentation. 


32 The Agency has employed a number of formal quality assessment mechanisms, but individual programs 
have not applied them consistently. We concluded that the mechanisms currently used do not, either individually 
or collectively, provide systematic, transparent information on the adequacy of quality management systems and 
practices in the Agency’s statistical programs or on the quality that they actually achieve. 


Background and other observations 


3 Statistics Canada is responsible for collecting, compiling, analyzing and publishing statistical information 
on the economic, social and general conditions of Canada and Canadians. The statistics that the Agency produces 
support the development, implementation and evaluation of policies, programs and decision making in all sectors. 
They help us make informed decisions about such matters as where to live, what careers to pursue and how to 
vote. The Agency is widely respected among its peers, and has an international reputation for independence, 
innovation and quality that is second to none. 


3.4 Rapid social, economic and other changes have heightened the demand for reliable, objective statistical 
information on a wide variety of issues. As the demand for and use of statistics grow, their quality becomes 
increasingly important. Statistics Canada has identified six characteristics that its systems and practices for 
managing quality need to address: relevance, accuracy, timeliness, accessibility, interpretability and coherence. 


525 The Agency carried out self-assessments of four major surveys for our audit. Each assessment reached a 
positive conclusion about the overall adequacy of quality management. The work was well planned and executed. 
We concluded that in three of the four surveys, the self-assessments provide reasonable assurance that quality 
management systems and practices are adequate. In the remaining case, however, we concluded that the 
weaknesses identified and the recommendations made are more important than the self-assessment suggests, and 
deserve the attention of senior management. 


3.6 While its policy on informing users of data quality and methodology is clear and well structured, the 
Agency’s implementation of the policy is inconsistent. Consequently, users are not always appropriately informed 
of the strengths and limitations of statistics. We also noted that while the quality of statistics figures prominently 
in its commitments to Parliament for results, the Agency’s most recent Performance Report, tabled in October 
1998, provides only limited information on the quality of the statistics that it produced. 


Statistics Canada’s responses to our recommendations are included in this chapter. The Agency either agrees 
to take action or notes that initiatives are under way in the case of six of the eight recommendations we 
make. In the remaining two cases, the Agency agrees with the intent of our recommendations and indicates 
that it will consider the issues further. 
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Introduction 


Statistics are essential to our society 


Ay | All levels of government, 
industry and other non-governmental 
organizations need statistics for a variety 
of purposes. These include monitoring the 
country’s economic and social conditions; 
developing performance information; 
planning and evaluating the policies, 
programs and investments of governments 
and the private sector; entering into labour 
contracts; supporting policy debates and 
advocacy; and keeping the public 
informed. 


3.8 Statistics are also used for 
statutory or regulatory purposes such as 
determining electoral boundaries; 
distributing federal funds to provinces; 
apportioning federal and provincial taxes; 
determining the eligibility of the 
unemployed for employment insurance; 
and indexing payments to beneficiaries 
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(for example, Canada Pension Plan 
payments). 


39 In addition, statistics about 
Canada’s economic and social conditions 
keep us informed. They help us in 
deciding where to live, what careers to 
pursue and what investments to make, for 
example. They also support our 
democracy by helping us make informed 
decisions about voting and other actions 
designed to influence governments. 


3.10 Rapid and accelerating social, 
economic and geopolitical changes have 
heightened the demand for reliable, 
objective statistical information on a wide 
spectrum of issues like the environment, 
health and provincial economies. As the 
demand for and use of statistics grow, 
their quality becomes increasingly 
important. 


As the demand for and 
use of statistics grow, 
their quality becomes 
increasingly important. 


Statistics Canada is responsible for 
Canada’s national statistics 


Rol Statistics Canada’s mandate 
derives mainly from the Statistics Act. The 
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Statistics Canada has 
put in place a wide 
variety of policies and 
processes to manage 
the quality of 
Statistics. 


3-8 


Act requires the Agency to collect, 
compile, analyze and publish statistical 
information on the economic, social and 
general conditions of the country and its 
citizens. Statistics Canada also has a 
mandate to provide professional 
co-ordination and leadership of the 
country’s statistical system. Consistent 
with this mandate, it identifies as its 
mission “to inform Canadian citizens, 
businesses and governments about the 
evolution of their society and economy 
and to promote a high quality national 
statistical system”. 


SZ With planned gross expenditures 
of almost $435 million in 1998-99 
(funded by a parliamentary appropriation 
of $360 million and revenues of 

$75 million), the Agency has some 

360 statistical programs and releases more 
than 1,000 statistical products each year. 
Its objective is to provide: 


..comprehensive and relevant 
statistical information on the 
economic, demographic and social 
structure of Canada in order to 
support the development, 
implementation and evaluation of 
policies, programs and 
decision-making. 


3.13 As noted in the most recent 
Annual Report to Parliament of the 
President of the Treasury Board 
(Managing for Results 1998), Statistics 
Canada has committed itself to furnishing 
Canadians with objective and non-partisan 
statistics. These statistics are intended to 
provide, for various aspects of Canada’s 
economy and society, measures that: 


e are of high quality, relevant to policy 
formulation and decision making, and 
responsive to emerging issues; and 


e contribute to fulfilling legal 
requirements, informing Canadians about 
current and emerging economic and social 
issues and maintaining a national 
infrastructure for statistical knowledge. 


3.14 In its view, Statistics Canada’s 
effectiveness depends on, among other 
things, the safeguarding of respondents’ 
confidentiality, the relevance of its 
programs, the quality and accessibility of 
its products, the attainment of high 
professional standards, and the control of 
the burden on survey respondents. 


The Agency is committed to producing 
statistics of high quality 


3.15. The quality of statistics thus 
figures prominently in Statistics Canada’s 
effectiveness and in its commitments to 
Parliament for results. The quality of any 
product or service is measured by how 
well it serves users’ needs and meets their 
expectations. From a user’s perspective, 
therefore, the quality of statistics is their 
“fitness for use”. It is a complex concept, 
which depends on certain fundamental 
characteristics of quality and on the 
intended uses of the statistics. 


3.16 While there is a general 
recognition among statistical agencies that 
quality is multi-dimensional, and some 
convergence of opinion about the range of 
characteristics that make up quality, there 
is no international standard definition for 
statistical quality. In common with other 
reputable statistical agencies, Statistics 
Canada approaches quality from the user’s 
perspective. Its Quality Assurance 
Framework document identifies six 
characteristics of quality that its policies 
and practices need to address: relevance, 
accuracy, timeliness, accessibility, 
interpretability and coherence (see 

Exhibit 3.1). 


3.17 Over the past 20 years, Statistics 
Canada has put in place a wide variety of 
policies and processes to: 


e ensure the ongoing relevance of its 
programs; 


e build quality into its programs and 
products through appropriate design, 
execution and the use of new 
technologies; and 
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¢ maintain an environment that 
encourages a concern for quality 
throughout the Agency. 


3.18 In 1981, for example, senior 
management of the Agency urged staff to 
follow a code of behaviour for statistical 
agencies with respect to quality assurance. 
Our 1985 follow-up of an earlier audit 
indicated that the Agency had taken a 
number of steps to improve its quality 
management processes. These included 
issuing comprehensive Quality 
Guidelines; revising its 1978 Policy on 
Informing Users of Data Quality and 
Methodology; establishing a National 
Statistics Council; and implementing a 
long-term planning process. These 
quality-related tools and mechanisms 
remain in effect, and others have since 
been introduced. 


3.19 Within the framework of the 
policies, guidelines and other initiatives 
that are in place, it is generally left to each 
program manager to select and implement 
the quality management techniques that 
are appropriate to their specific programs. 


Focus of the audit 


3.20) We examined Statistics Canada’s 
systems and practices for assessing the 
adequacy of quality management inits _ 
Statistical programs, and for reporting to 
users on the quality of its statistics and to 
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Parliament and the public on its 
performance. 


3.21 In view of the technical 
complexities involved and the 
professional expertise available within the 
Agency, we proposed that it carry out 
self-assessments of a number of statistical 
programs as an integral part of our audit, 
focussing on the adequacy of systems and 
practices for managing quality. The 
Agency agreed to this proposal and 
completed assessments of four of its major 
surveys. 


S22 Our objectives in this audit were 
to determine whether Statistics Canada: 


e systematically assesses the adequacy 
of quality management systems and 
practices in its statistical programs to 
ensure that it has the information it needs 
to manage and report on quality; 


e can provide reasonable assurance, 
based on its self-assessments, that quality 
management systems and practices in the 
four selected surveys are adequate; and 


e appropriately informs users about the 
quality of data and the methodology used 
to develop the statistics. 


3.23 Further details on the audit can 
be found in About the Audit at the end of 
the chapter. 


Accessibility: A characteristic relating to the availability of information from the holdings of the 


Agency. 


Accuracy: The degree to which data correctly estimate or describe the quantities or characteristics 
that the statistical activity was designed to measure. 


Coherence: The degree to which data and information from a single statistical program, and data 
| brought together across data sets or statistical programs, are logically connected and complete. 


Interpretability: The ease with which a user may understand and properly interpret data or 


| 
| information. 


Timeliness: The length of time between the availability of information and the event or 
phenomenon it describes, considered in the context of the time period that permits the information 


to be of value and still acted upon. 


Relevance: A qualitative assessment of the value contributed by data. 


Exhibit 3.1 


The Six Characteristics of 
Quality - Statistics Canada’s 
Perspective 


Source: An Outline of 
Statistic Canada’s Quality 
xt Assurance Framework 
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Managing quality 
across or within 
statistical programs is 
an ongoing balancing 
act. 


Observations and 
Recommendations 


Corporate Assessment of Quality 


Information on quality is required for 
management and accountability 


3.24 Because of constantly changing 
user needs and the complex nature of 
quality, it is an ongoing balancing act to 
manage quality across statistical programs 
as well as in any single program. For 
example, in a climate of fiscal restraint, 
demands for new statistical series may 
require the acceptance of lower quality in 
some existing series. Similarly, meeting 
demands for increased timeliness in a 
statistical program may involve some 
trade-off with other characteristics of 
quality, such as accuracy. 


3.25 In addition to this internal 
balancing act, the Agency 1s committed to 
external reporting. It reports to Parliament 
and the public on its performance, and to 
users on the quality of its statistics. We 
believe that Statistics Canada thus needs 
to have meaningful information on the 
adequacy of its quality management 
systems and practices (including 
information on the quality of its statistics) 
to help improve its statistical programs, 
support corporate decision making and 
assurance, and report externally on its 
performance in achieving quality. 


3.26 Thus, we expected that the 
Agency would systematically assess the 
adequacy of its quality management 
systems and practices in individual 
statistical programs, and assess the quality 
actually achieved. Such assessments, 
reported at the corporate level, are 
particularly important given the wide 
latitude managers have to select and 
implement quality management 
techniques in individual programs. 


A number of assessment mechanisms 
have been used 


3.27 We found that, like other 
statistical agencies, Statistics Canada has 
used a number of different mechanisms to 
assess quality. Appendix A provides a 
brief overview of approaches to quality 
assessment and reporting that we observed 
in some of the other statistical agencies 
we visited. 


3.28 Program evaluations. In the 
1980s the Agency established a corporate 
program evaluation function. Over a span 
of five years, it completed 19 evaluations 
of statistical programs, drawing heavily on 
feedback from major users. The 
evaluations made some 200 
recommendations for improvements to 
help meet user needs; many of the 
recommendations were implemented. In 
1991-92, following an assessment of this 
first cycle of evaluations, the Agency 
devolved responsibility for evaluation to 
program managers (see paragraph 3.31). 
There have been no corporate program 
evaluations by the Agency since then. 


3.29 Internal audits. In 1990 the 
Agency carried out an internal audit of 
compliance by individual statistical 
programs with the Agency’s Policy on 
Informing Users of Data Quality and 
Methodology. It found that many of the 
audited programs did not fully comply 
with the Policy, and proposed that a 
follow-up audit be carried out two to three 
years later (that is, in 1994-1995). In view 
of other priorities, the follow-up was 
postponed to 1998. After we set out our 
plans for this audit, the Agency decided 
not to proceed with the follow-up. 
Although the Agency retains an internal 
audit function, we found that in recent 
years it has carried out no audits focussed 
specifically on managing the quality of 
Statistics. 


3.30 Program Reports. As a means of 
devolving responsibility for evaluation to 
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program managers, in 1991—92 the 
Agency introduced a requirement that 
each program produce a detailed annual 
Program Review Report as input to the 
Agency’s Long-term Planning Process. A 
subsequent change required Program 
Reports to be produced every two years. 
The Program Reports are designed to be 
self-evaluation reports from program 
managers to the Chief Statistician of 
Canada on the achievement of program 
objectives. According to the planning 
guidelines, the reports should pull together 
from various sources the key findings on 
program performance and include 
extensive indicators of quality. 


3.31 Although these Program Reports 
have been used in the Agency’s 
Long-term Planning Process, we found 
that they have not always been submitted 
on time. Twelve of the 19 product-related 
Program Reports scheduled for the first 
two-year cycle had been submitted when 
we completed this audit, and only one of 
the 15 scheduled for the second cycle. 
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important implications for the assessment 
and reporting of statistical quality (see 
Exhibit 3.2). 


3.32 We also found that when Program 
Reports were submitted, they did not 
always include quality indicators as 
described in the guidelines for preparing 
these reports. Finally, we found that some 
Reports did not cover all the programs for 
which the reporting manager was 
responsible. We note, however, that some 
of the reporting managers are responsible 
for a great many programs — more than 
60 in one case. 


3.33 User input and feedback. 
Statistics Canada uses a variety of 
mechanisms to obtain user input and 
feedback on its statistical outputs. These 
help it to keep its programs relevant and 
to meet the evolving needs of those who 
use its products. The mechanisms include: 


e the National Statistics Council, 
which advises the Chief Statistician on 
program priorities and the full range of the 


One reason why the Reports are late is 
that many of the programs are being 
revised under the Agency’s ongoing e a Federal-Provincial-Territorial 
Project to Improve Provincial Economic Consultative Council on Statistical Policy, 
Statistics (PIPES). That project itself has and its sub-committees; 


Agency’s activities; 


| On 23 October 1996, the governments of Canada, Newfoundland, Nova Scotia and New Brunswick Exhibit 3.2 

agreed to combine their individual sales taxes into a single “Harmonized Sales Tax”. Since | April 

| 1997, there has been a single sales tax rate of 15 percent in the three provinces instead of three 

| separate provincial sales taxes and a distinct federal Goods and Services Tax. Revenue Canada 

collects the revenues from the combined taxes, and the final allocation among the four 

| governments is determined on the basis of a formula using aggregate statistics compiled by 
Statistics Canada. 


Project to Improve Provincial 
Economic Statistics (PIPES) 


The quality of the statistical information used for this allocation must meet demanding 

| requirements. In February 1997, the Treasury Board authorized Statistics Canada to proceed with 

| the Project to Improve Provincial Economic Statistics (PIPES). A total of $74.5 million and 
780 full-time equivalents were allocated for PIPES over two years, 1997-98 and 1998-99. The 
objective is to measure the final sale of goods and services accurately by province, in sufficient 
detail and in a timely manner. A combination of administrative records and improved business and 
household surveys will be used to produce the required information. Each piece of information will 
be assessed within the framework provided by a detailed set of provincial Input-Output Accounts. 


To demonstrate quality improvements to stakeholders, Statistics Canada has undertaken to develop 
concrete measures of quality for these statistics. Various types of quality information, indicators 
and profiles are to be developed. A data quality analysis is to be carried out annually and the 
information on quality will be tracked from year to year. 


Source: Statistics Canada 
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The Agency’s formal 
mechanisms do not 
provide systematic 
information on the 
adequacy of quality 
management 
processes or on 
quality achieved. 


e professional advisory committees in 
major subject areas; 


e bilateral relationships with key 
federal departments and agencies; 


e avariety of ad hoc task forces; and 


e surveys and studies of user 
satisfaction. 


3.34 The Statistical Data 
Documentation System. In the early 
1980s the Agency implemented the 
Statistical Data Documentation System 
(SDDS), an information system to support 
planning, evaluation and marketing as 
well as other management functions. The 
SDDS was designed to contain detailed 
information on each survey (for example, 
its content, coverage, design and 
methods). 


3.35 We found that the information 
entered into the system has been little 
used for evaluation and planning. In 
recent years the SDDS has been used 
mainly to produce the Guide to Statistics 
Canada’s Programs and Products, the first 
document that many users consult for 
information about the Agency’s products. 


3.36 Program managers submit annual 
updates to the SDDS, but SDDS staff do 
not verify their completeness. A “data 
quality” section for each survey is 
required to describe briefly the most 
important sources of error, and to provide 
quantitative measures and qualitative 
descriptions of data quality. However, we 
were told that one in every six surveys had 
no information about data quality in the 
SDDS. Further, our review showed that 
where such information was available, it 
often failed to describe quality in a 
consistent and meaningful way. 


Formal assessment mechanisms are not 
applied consistently 


3.37 We recognize that the Agency 
has a number of informal processes for 
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assessing and monitoring quality, 
including personal knowledge and verbal 
presentations. Nevertheless, in 
government documents and other sources 
it has emphasized the importance of 
formal and interrelated processes 
(paragraphs 3.28—3.36) for documenting 
and monitoring performance, including 
quality. 


3.38 As already noted, Program 
Reports, one of the Agency’s key formal 
mechanisms currently used for evaluating 
programs, are frequently not submitted on 
time and may lack some of the required 
information. Although valuable in their 
own right, other formal mechanisms (such 
as internal audits, the processes related to 
the SDDS, and processes for gathering 
client input and feedback) have not filled 
this gap. 


3.39 We concluded that the Agency’s 
formal quality assessment mechanisms are 
not applied consistently. As a result, they 
do not provide, either individually or 
collectively, systematic and transparent 
information on the adequacy of quality 
management systems and practices in the 
Agency’s statistical programs or on the 
quality they actually achieve. 


3.40 Statistics Canada should ensure 
that formal quality assessment 
mechanisms are applied consistently so 
that they provide systematic 
information on the adequacy of quality 
management systems and practices in 
individual statistical programs and, to 
the extent possible, information on the 
quality that they achieve. 


Agency’s response: We agree with the 
need to improve compliance related to 
some of our internal reporting 
mechanisms. However, we maintain that 
this lack of compliance has not in any way 
Jeopardized the quality of our statistical 
output. We have already launched an 
initiative to improve internal reporting. 
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Statistics Canada’s 
Self-Assessment of Four Surveys 


The Agency assessed the adequacy of 
quality management systems and 
practices 


3.41 Statistics Canada agreed at the 
outset of our audit to carry out 
self-assessments of four major surveys as 
a one-time project. The objective was to 
assess the adequacy of quality 
management systems and practices in the 
four surveys. The work was carried out 
under the Quality Assurance Framework 
that the Agency had documented before 
undertaking the assessments, and in the 
light of the priorities and resources that 
management had allocated to the surveys. 


ee 
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3.42 We selected the surveys jointly 
with Statistics Canada, based on their 
significance to users and their public 
profile, as well as the range of subject 
matter and data collection techniques they 
represented. Exhibit 3.3 provides brief 
descriptions of the selected surveys: the 
Consumer Price Index, the Labour Force 
Survey, the Monthly Survey of 
Manufacturing and the Uniform Crime 
Reporting Survey. 


The self-assessments were well planned 
and executed 


3.43 The self-assessments were 
conducted in two stages. As a first step, 
Program Divisions were asked to 
document the quality management 
activities for the selected surveys. This 


Exhibit 3.3 


Consumer Price Index (CPI): This survey provides a measure of the rate of price change of 
goods and services bought by Canadian consumers. The CPI is obtained by comparing, over time, 
the cost of a fixed “basket” of commodities purchased in a particular year. It is the most widely 
used indicator of price changes in Canada, and is often used as a measure of inflation (or 
deflation). It is used for adjustments to social and welfare payments, rental agreements, spousal 
and child support payments, and other forms of contractual and price-setting arrangements (for 
example, cost-of-living adjustments and labour contracts governing wages). 


Labour Force Survey (LFS): This is a monthly survey that collects data on the labour market 
activities and demographic characteristics of working-age Canadians. It is also a vehicle for 
supplementary surveys of the labour market. The LFS started in November 1945 and has evolved 
over the years, with major changes to its content and sample design in 1976 and 1997. Data are 
collected from a sample of 52,350 households, involving some 110,000 respondents. The sample 
has been designed to be representative of working-age Canadians, with the exception of residents 
of the Yukon and the Northwest Territories, persons living on First Nations reserves, inmates of 
institutions and full-time members of the Armed Forces (these exclusions account for about 

3 percent of the population). 


Monthly Survey of Manufacturing (MSM): This survey collects information on the value of 
shipments, inventories, and unfilled and new orders in Canada’s manufacturing industries. Data 
since 1947 are available. It is used to monitor the business cycle in the key manufacturing sectors 
of the economy, to evaluate and develop financial and industrial policy, and to provide market data 
at the detailed industry level. The MSM sample consists of approximately one third of 
establishments in manufacturing industries in Canada. The survey questionnaire, sample frame, 
and processing system are currently being redesigned. 


Uniform Crime Reporting (UCR) Survey: This survey collects data on the nature and extent of 
crime in Canada. In 1962, the UCR Survey began collecting aggregate data on reported crime 
substantiated through police investigation. The survey is based on the administrative records 
maintained by police forces. Revisions in 1983 resulted in the phased implementation of 
incident-based data collection, starting in 1988 (UCR2). Since then, police forces have continued 
to convert to the UCR2. The UCR2 Survey currently has 180 police forces reporting to it whose 
data represent about half of the national volume of reported crime. UCR2 data are converted to 
obtain national coverage at the level of aggregate estimates. 
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Brief Descriptions of the Four 
Surveys Assessed by 
Statistics Canada 


Source: Statistics Canada 
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We reviewed the 
self-assessments of 
four major surveys 
that Statistics Canada 
carried out for our 
audit. 


Statistics Canada conducted 
self-assessments of quality 


management in four major surveys 


(see paragraphs 3.41—3.48) 


recognized the fact that each Program 
Division possesses a unique set of 
knowledge and expertise. 


3.44 The Agency developed a set of 
guidelines to help each Program Division 
describe systematically the way it 
manages quality in the planning, design, 
execution and analysis phases of the 
survey, as well as in its overall 
management. Exhibit 3.4 shows the types 
of information the Program Divisions 
were asked to provide. 


3.45 The second step in the 
self-assessment involved review teams of 
experienced Statistics Canada staff drawn 
from outside the program areas. They 
assessed the information and reached 
conclusions about the adequacy of quality 
management systems and practices. The 
review teams provided a degree of 
independence to help ensure an objective, 
fair and equitable review process. A 
senior-level Steering Committee provided 


guidance and oversight to the work of the 
review teams. 


3.46 The self-assessment process 
began in January 1998, with the Uniform 
Crime Reporting Survey serving as a pilot. 
The pilot provided feedback and evidence 
on the viability of the approach. On the 
basis of this pilot, the self-assessments of 
the other surveys were initiated in May 
1998. Reports documenting the results of 
the self-assessments were available to us 
by early September 1998. 


3.47. Throughout the process, we 
maintained close contact with the 
Agency’s Steering Committee and the 
review teams. We reviewed the planning 
and execution of the self-assessments, 
reviewed the supporting documentation 
and held follow-up discussions with the 
Agency’s review team and program 
managers as well as interviews with users 
of statistical products. 


3.48 Overall, we found that the 
self-assessments were carried out in 
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prices and 
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i Statistics 
199! pa Catalogue 0, 31-001-XPB 


Monthly 


July 1997 


Ben sr soci 


Bol Seti Statistique 


Caaingue na. 71-201-XPB. N*71-201-XPB aucatadogue 
| Historical Statistiques 
Catalogu pa 86.296:XPE hw | Labour chronologiques 


Canadian mM Survey of 
— | Manufacturing manufacturiéres 


N*62-010-XPS au catalogue 

Prix ala 
consommation et 
indices des prix 


Force sur la population 
Statistics active 
ae 7 


NY91-001-XPB au catalogue 


Enquéte mensuelle | 
sur les industries 


Juillet 1997 


Report of the Auditor General of Canada — April 1999 


accordance with the methodology and 
schedule agreed to with our Office. The 
descriptions prepared by Program 
Divisions provided the background 
information needed to understand the 
programs in general and the management 
of their quality in particular. The review 
teams were independent of Program 
Divisions and knowledgeable about 
quality management as well as about the 
surveys they were reviewing. We 
concluded that the self-assessments were 
generally well planned and executed. 


The conclusions of three of the four 
self-assessments were consistent with 
the evidence 


3.49 The following excerpts show that 
the self-assessments reached positive 
conclusions about the overall adequacy of 
quality management systems and practices 
in each of the four surveys: 


e Consumer Price Index (CPI). “...in 
spite of some inherent weaknesses that are 
not unique to Canada, the CPI remains an 
indicator which is relevant, reliable and 
well in line with the needs of its key 
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users... [The Agency] has put proper 
mechanisms in place to continuously 
improve this widely used macro-economic 
indicator.” 


e Labour Force Survey (LFS). “LFS 
processes for monitoring and assuring 
quality are appropriate given the profile of 
the program and the resources that are 
available to carry it out.” 


e Monthly Survey of Manufacturing 
(MSM). “...the processes for ensuring the 
maintenance of quality are adequate for a 
survey of its prominence and given its 
budget.” 


e Uniform Crime Reporting Survey 
(UCR). “Given the nature of the program 
and its objectives, budget and constraints, 
no major changes to the overall quality 
management and quality assurance 
program...are being recommended. Some 
improvements in specific areas could be 
made without significant cost. These are 
matters of ‘fine tuning’.” 


3.50 In three of the four cases, we 
found sufficient appropriate evidence to 
support these overall conclusions. We 
concluded that the self-assessments 


e A description of the survey, its goals and objectives, the major uses and users, the methodology, 


| 
its data products, costs and sources of funding. 
| 


e A description of the evolution of the survey, and the context in which these changes were made. 


handle discrepancies. 


response or imputation rates. 


e How the relevance of the survey is ensured, including feedback mechanisms to assess the level 
of satisfaction of major users with data products. 


e Overviews of the latest redesign and research activities affecting the survey. 
e A description of the steps that are built into the design and implementation to ensure quality. 


e An outline of mechanisms imbedded in the survey to check for consistency and coherence and 


e A description of the activities involved in monitoring quality measures, such as sampling error, 


e An outline of the survey management structure, both for ongoing operations and development 


activities (redesign or re-engineering). 


e A description of how decisions on changes in the scope, methodology, or level of funding 
affecting the survey are made, including all review mechanisms within and outside Statistics 


Canada. 


e A description of how corporate initiatives in areas such as human resource management are 


affecting the survey. 


e Any current important issues that can affect the relevance or credibility of the Program. 


Exhibit 3.4 


Information Provided by 
Program Divisions for 
the Self-Assessments of 
the Surveys 


Source: Statistics Canada 
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provided reasonable assurance that quality 
management systems and practices are 
adequate in the Consumer Price Index, 
Labour Force Survey and Monthly Survey 
of Manufacturing. In the case of the 
Uniform Crime Reporting Survey, we 
believe the weaknesses identified, and the 
recommendations made, are more 
important than the self-assessment 
suggests. 


3.51 As a component of the National 
Justice Statistics Initiative, the purpose of 
the UCR Survey is to help ensure that the 
Canadian public has accurate information 
on the nature and extent of crime in 
Canada. The UCR Survey relies on 
administrative data provided by police 
forces across the country. As a result, 
Statistics Canada has less direct control 
over the completeness and the quality of 
the data than in those cases where it 
collects survey data itself. The two key 
factors that affect the quality of the UCR 
statistics, therefore, are the participation 
of all police forces in Canada and the 
integrity of the record systems maintained 
by them. The self-assessment clearly 
reports quality weaknesses in both these 
areas. 


3.52 An additional complicating factor 
is that the UCR Survey relies on two 
conceptually different surveys that run in 
parallel. Some police forces report 
summary data on the number of offences 
(UCR1). Others provide detailed data on 
individual criminal incidents, including, 
for example, the age and sex of victims 
and offenders, the level of injury, weapons 
involved, location of incident and dollar 
values of property and drug crimes 
(UCR2). Statistics Canada converts the 
UCR2 data to summary data and 
combines them with UCRI to publish 
crime statistics at the national level. In 
addition, it publishes the UCR2 data that 
are available. Although considerably 
richer for purposes of analysis, the 
published UCR2 data are limited by the 
fact that they cover only about half the 


reported crime in Canada and are not 
representative of Canada as a whole. 


3.00 In our view, these circumstances 
place a particular onus on the Agency to 
inform potential users as clearly as 
possible about the quality of the UCR data 
and limitations on their use. We note that 
while the UCR review team made 
recommendations to strengthen practices 
for managing the relevance and 
interpretability of the survey (see 
Appendix B), its overall conclusion states 
that “no major changes are being 
recommended” and labels the 
recommendations as “fine tuning”. 
Although the meaning of terms such as 
“major changes” and “fine tuning” are 
clearly matters of judgment, we believe 
the recommendations are more important 
than the self-assessment suggests and 
deserve the attention of senior 
management. 


3.54 Appendix B includes excerpts 
that summarize the self-assessments’ 
conclusions about the four surveys under 
each of the six characteristics of quality, 
along with our comments. In most cases, 
we agree with the review teams’ 
conclusions and with the opportunities for 
improvement that they identified. In some 
cases, we point to additional areas where 
we believe that quality management 
practices, including the reporting of 
quality, can be improved. 


A wider application of the 
self-assessment technique merits 
consideration 


3.55 Although Statistics Canada 
carried out the four self-assessments as a 
one-time exercise for our audit, our 
review of the process and the results of the 
assessments suggest that a wider 
application of this technique merits 
consideration — especially as the Agency 
does not now have other independent 
assessment mechanisms in place. If the 
Agency were to apply the technique more 
broadly, it would need to ensure that 
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assessments were carried out rigorously 
and that their conclusions were clear. 


3.56 The constraints on the 
self-assessments carried out for this audit 
would need to be removed. The four 
self-assessments set out to examine the 
adequacy of quality management practices 
explicitly within the context of the 
priorities and resources allocated to the 
programs by corporate management. 
Consequently, the assessments looked 
only at whether the programs did what 
could reasonably be expected in 
prevailing circumstances to manage 
quality. In addition, they did not set out to 
assess or to comment on the quality 
actually achieved in the four surveys. 


S57 We believe that a stronger focus 
on results would enhance the 
self-assessment technique. The following 
kinds of issues would need to be assessed: 


e the appropriateness of survey design 
parameters, including quality targets as 
appropriate; 


e the extent to which design 
parameters have been achieved; 


e changes in quality since the last 
assessment; 


e the nature and extent of reporting on 
quality to users and the public; 


¢ documentation of key decisions 
affecting quality, and rationales for such 
decisions; and 


e the adequacy of methods to control 
the quality of data from external sources. 


3.58 We believe that a wider 
application of self-assessments would 
likely also require some training of 
reviewers in assessment techniques, as 
well as documented guidance for review 
teams. 


3.59 Statistics Canada should review 
the potential for a wider application of 
an enhanced self-assessment technique 
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as one component of the formal 
processes it uses to assess quality. 


Agency’s response: We will consider the 
benefits and costs of applying this 
technique as one component of our quality 
management approach. 


Reporting Performance to 
Parliament 


The Agency’s Performance Report to 
Parliament provides limited 
information on quality 


3.60 The government reformed its 
Expenditure Management System in 1995. 
Part of that reform included providing 
better planning and performance 
information to Parliament. Departmental 
Performance Reports are now tabled in the 
fall to start the cycle of budget and 
business planning decisions. Among other 
things, these reports are to provide 
information on results achieved, important 
management initiatives and financial 
performance for consideration by 
parliamentarians in the Estimates and 
Supply process. Reports on Plans and 
Priorities, which reflect decisions based 
on performance and government priorities, 
are tabled in the spring. 


A stronger focus on 
results would enhance 
the self-assessment 
technique. 


3.61 As already noted, the Agency has 
committed itself to providing Canadians 
with high-quality statistics that are 
relevant to policy making and responsive 
to emerging issues. In many of its policies 
and processes it gives a central place to 
the quality of statistics. We therefore 
expected to find quality-related 
performance indicators in the Agency’s 
Performance Report for the period ended 
31 March 1998, which was tabled in 
October 1998. 


3.62 Under the heading “Information 
Quality”, the Performance Report notes 
that Statistics Canada uses a wide range of 
quality assurance practices and conducts 
intensive “institutional” quality 
verification of all data releases. Under the 
same heading, the Report notes that 
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The Agency’s most 
recent Performance 
Report provides no 
performance 
information about the 
accuracy, 
interpretability or 
coherence of the 
Statistics it produced. 


indicators of data quality are included in 
all publications, although our review, and 
the Agency’s own assessments, showed 
that this is not always the case (see 
Informing Users About Data Quality 
and Methodology on this page). 


3.63 The Performance Report includes 
some descriptions of processes used to 
keep programs relevant — including 
external advice and user feedback — and 
provides examples of changes made as a 
result. With respect to timeliness, the 
Report shows the time elapsed between 
the reference period and the release dates 
for six major surveys; but it provides no 
rationale for selecting only these surveys 
from among some 360 statistical 
programs. For 28 selected outputs in 
“major subject areas’, the Report includes 
the frequency of publication (for example, 
monthly or quarterly) and indicates 
whether they were released on schedule in 
1997-98. 


3.64 The only other quality-related 
indicators in the Report are indicators of 
accessibility. They provide some 
information about the growing use of the 
Agency’s Internet site, as well as changes 
to the statistical information available 
there. 


3.65 Overall, we concluded that the 
Performance Report contains only partial 
information about the Agency’s actual 
performance in terms of the six 
characteristics it has identified as central 
to quality. In particular, the Report 
provides no performance information at 
all about the accuracy, interpretability or 
coherence of statistics. 


3.66 We were told that the process 
used to gather information for the 
Performance Report was informal and not 
linked explicitly to other quality-related 
initiatives. Program Reports were 
consulted to some extent. However, no use 
was made of the Statistical Data 
Documentation System, which could be a 
key database to generate quality-related 
indicators for external reporting. 


3.67 Statistics Canada should 
improve the coverage and content of 
information on statistical quality in its 
annual Performance Report to 
Parliament by drawing on quality 
information available from internal 
assessment and reporting systems. 


Agency’s response: We will review the 
content of the annual Performance Report 
to Parliament to improve the way in which 
information on Statistical quality is 
presented. However, we do not believe that 
it is possible to produce simple summary 
quality measures across a wide variety of 
programs in a way that is useful and 
meaningful for Parliament. 


Informing Users About Data 
Quality and Methodology 


3.68 All statistics are, to some extent, 
estimates of the reality they seek to 
convey. Therefore, they have to be used in 
full awareness of their strengths and 
limitations. Unlike users of many products 
and services, however, users of statistics 
cannot readily assess all of the important 
quality characteristics of the data 
produced by Statistics Canada. For the 
most part, users have to rely on the 
Agency’s integrity and professionalism, as 
well as what it tells them about data 
quality and the methodology used to 
collect and compile the data. 


3.69 Statistics Canada recognizes the 
need to provide potential users with 
information about data quality and the 
concepts, definitions and methods used, so 
they can determine whether the statistics 
fit their purposes and can make informed 
use of them. Since 1978, the Agency has 
had a Policy on Informing Users of Data 
Quality and Methodology. 


A 1990 internal audit found inconsistent 
practices 


3.70 As we have noted, in 1990 the 
Agency carried out an internal audit of its 
compliance with this policy on informing 
users. In a sample of 20 “major” and 
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50 “other” surveys, the audit identified 
many instances of non-compliance. 


me A The internal audit report made a 
number of important observations and 
recommendations on Statistics Canada’s 
dissemination of quality-related 
information to users. For example, the 
report stated that: 


e as aminimum, an explicit reference 
to a source of information on methods, 
concepts and definitions should be a 
mandatory requirement in all catalogued 
publications; 


e consideration should be given to 
requiring a standard statement in each 
publication to the effect that “the data 
included in this publication are subject to 
various coverage, response [and other] 
errors’, as applicable; 


e every product should undergo a 
low-cost annual assessment and 
certification process using existing data 
quality indicators as a minimum. The 
results should be summarized or 
referenced in the statistical product; and 


e where no quality assessment or 
supportable statement on major aspects of 
quality is available, this should be stated 
clearly in the related publications. 


The Agency’s Policy is clear and well 
structured 


3./2 In 1992 the Agency updated its 
Policy on Informing Users of Data Quality 
and Methodology and, in doing so, 
implemented many of the 
recommendations of the 1990 internal 
audit. The key elements of the current 
Policy are set out in Exhibit 3.5. We 
assessed the Policy and found it to be well 
structured. It sets out clearly the 
expectations for program managers, both 
mandatory and discretionary. 


ey fe We compared Statistics Canada’s 
Policy with the approaches taken by 
reputable statistical agencies in other 
countries. We found that while all of these 
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agencies recognized the need to inform 
users of data quality and methodology and 
had taken steps to do so, not all of them 
had documented policies in place. The 
policies that we saw varied in their 
structure and content. Although some 
were quite similar to that of Statistics 
Canada, none, in our view, were more 
advanced. 


The Agency’s practices in informing 
users continue to be inconsistent 


3.74 As noted in paragraph 3.29, the 
Agency decided not to carry out the 

follow-up of the 1990 internal audit that 
had been planned for 1998. We therefore 
carried out a limited test of its disclosure 


The Agency has a 
clear policy on 


practices. informing users of 
3.75 We assessed a selection of 10 data quality and 
products to determine the nature and + 
extent of disclosure about data quality and methodology, but itis 
methodology both in hard copy not applied 
publications and electronic media. We consistently. 


used criteria based on the 11 mandatory 
minimum requirements specified in the 
Agency’s current Policy on Informing 
Users of Data Quality and Methodology 
(see Exhibit 3.5). 


3.76 Our test showed that disclosure of 
information on data quality and 
methodology in the Agency’s products did 
not always comply with the mandatory 
minimum requirements of the Policy. 
Therefore, users are not always 
appropriately informed of the strengths 
and limitations of statistics. Disclosure 
was inconsistent across products. 
Specifically, we noted the following: 


e Disclosure of data quality and 
methodology is not clearly or consistently 
identified as such. For example, data 
quality is often discussed in sections titled 
“Note”, “Technical Notes” or “Survey 
Overview”. 


¢ Information is generally more on the 
definition of errors or description of 
quality control processes than on actual 
quality achieved. Few quantitative 
measures of quality are provided. 
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¢ Coefficients of variation are often BRIA) We found also that disclosure was 
the only indicators of quality given. While —_ inconsistent across media of 
coefficients of variation may be helpful dissemination. In particular, information 


for technical users, many other users have _ available on or through the Agency’s 
difficulty understanding them. We believe Internet site showed weaker compliance 


that users need also to be provided with than printed publications. 

simple, understandable statements or 

indicators of data quality whenever 3.78 To ensure that its Internet site 
possible. complies with its policy on informing 


Exhibit 3.5 


Policy on Informing Users of Data Quality and Methodology - Key Elements 


Section Contents 
Policy Five statements spell out the intent of the policy: 
1. Indicators of data quality and methodology are made available to users. 
2. Documentation must conform to standards and guidelines. 
3. Products are accompanied by or refer to such documentation. 
4. Exemption must be approved. 
| 5.  Cost-recovery surveys are encouraged to follow the same practices. 
Responsibilities Individual Programs 


— Inform users of the availability of information on data quality and methodology. 

— Disseminate existing measures or descriptions of data quality and documentation on 
methodology. 

— Implement procedures to generate information on data quality. 

Methods and Standards Committee 

— Report periodically on the state of compliance with the Policy. 

— Provide standards and guidelines to Programs. 

— Approve applications for exemption from the Policy requirements . 


General Principles General principles governing implementation are provided. 


The Standards Eleven mandatory minimum requirements are specified: 
1. An explanation of fundamental concepts. 
2. Basic definitions, including: 
— _ universe and target population; 
— key characteristics; 
— key indicators or data being disseminated. 
3. Time frame or reference period. 
4. Advice that data are subject to error. 
5. Advice that data may be subject to revisions. 
6. Advice that data may not be comparable over time. 
7. Coverage error or data quality rating. 
8. Sampling error for key characteristics. 
9. Response rate and extent of imputation. 
10. For administrative data, overall quality rating. 
11. Potential sources of error. 


The Guidelines Guidelines on how to provide additional information on data quality to all users and for 
different types of data. 


Source: Policy on Informing Users of Data Quality and Methodology, 1992 
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users, the Agency recently began to 
develop an Integrated Meta Database to 
replace the SDDS. It is to contain 
information on the data quality, concepts 
and underlying methodology of each 
Agency survey, and is to be accessible to 
all Internet users. Because the new 
database was still being developed at the 
time of our audit, we were unable to 
assess the nature and extent of the 
information it contains. 


3.79 Our findings on the lack of 
information and the inconsistencies in 
disclosure are in accordance with the 
Agency’s findings on “interpretability” in 
the four self-assessments it carried out 
(see Appendix B). 


3.80 Statistics Canada’s Methods and 
Standards Committee has functional 
responsibility for the Policy on Informing 
Users of Data Quality and Methodology, 
and is mandated to monitor its 
implementation. However, we found that 
the Committee has not done so, nor has it 
produced the periodic reports on the state 
of compliance that the Policy requires. We 
noted, too, that while program managers 
can ask the Methods and Standards 
Committee for exemptions from 
compliance with the Policy, no such 
exemptions have been requested or 
granted. 


3.81 The Agency’s Guide to Statistics 
Canada’s Programs and Products includes 
some information on data quality and 
methodology. However, our review 
showed that practice with respect to the 
inclusion of such information is 
inconsistent across programs. In some 
cases no information is provided; in 
others, information on the source and 
definition of possible errors is included. In 
still others, actual indicators of data 
quality are provided. Although the policy 
on informing users does not apply to the 
Guide, including information on data 
quality and methodology could help the 
many users who would consult the Guide 
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first when seeking information about the 
Agency’s products. 


3.82 Officials in the other statistical 
agencies we visited indicated that their 
own quality disclosure practices were also 
inconsistent. Most officials acknowledged 
that they could, and should, do better. 
Disseminating statistics by means of new 
technologies, such as the Internet and 
compact discs, was widely recognized as a 
particular challenge. 


3.83 Statistics Canada should ensure 
that its Policy on Informing Users of 
Data Quality and Methodology is 
applied consistently across products and 
dissemination media. 


Agency’s response: Agreed. 


3.84 In informing users of data 
quality and methodology, Statistics 
Canada should: 


e use a clear and standardized 
heading in all printed and electronic 
documents for a section that provides 
information on data quality and 
methodology, or that refers the user to a 
source where such information can be 
found; and 


e establish a flexible disclosure 
framework to define the contents of the 
data quality and methodology section in 
individual statistical products and 
related documents, to ensure that the 
intent of the Policy is met. The 
framework should allow for variations 
in the depth and breadth of the 
information provided, depending on the 
program, product or medium of 
dissemination. 


Agency’s response: Agreed 

An Integrated Approach to 
Managing Quality 

Many building blocks are in place 


3.85 After-the-fact assessments of 
quality cannot replace measures that build 
quality into statistical programs — 
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We believe there is a 
need for each 
statistical program to 
demonstrate regularly 
that it has met design 
parameters or quality 
targets. 


appropriate design and execution, the use 
of suitably qualified and motivated staff, 
the effective management of resources and 
activities, and a corporate culture of 
integrity. We support the Agency’s 
emphasis on building quality in through 
these and other means, and its approach to 
giving managers discretion to use the best 
techniques available to pursue quality in 
the Agency’s diverse statistical products. 


3.86 However, we believe there is a 
need for each statistical program to 
demonstrate regularly — through 
self-reporting, some form of independent 
assessment, or both — that it has met 
design parameters or quality targets, such 
as sample size and response rate (see 
paragraphs 3.40 and 3.59). The frequency 
and depth of such reporting would need to 
be decided with due consideration to the 
cost, complexity and importance of the 
statistical programs concerned. Without 
systematic quality assessments, it is 
questionable whether the Agency can 
assure itself, or others, that the systems 
and practices it uses to build in quality are 
effective. 


3.87 The Agency has many 
quality-related policies, guidelines and 
systems that could become the basis for 
effective quality assessment and reporting. 
Many of the necessary building blocks are 
already in place, and recent experience 
with the self-assessment approach may 
point to a useful addition. However, we 
believe that the Agency needs to reshape 
some of these building blocks and reorient 
others to make them more cohesive. A 
basis for such integration could be the 
newly documented Quality Assurance 
Framework, which sets out what the 
Agency considers to be the key 
characteristics of quality in statistics, and 
describes the processes already in place to 
manage quality. 


Better integration and documentation 
are needed 


3.88 The Agency’s quality-related 
initiatives were developed over a long 


span of time by different units in the 
organization. For example, the Statistical 
Data Documentation System was 
developed by the Standards Division; the 
Quality Guidelines by a team of 
professionals drawn from various units; 
the Policy on Informing Users of Data 
Quality and Methodology by the Methods 
and Standards Committee; and the 
Performance Report and guidelines for 
Program Reports by the Corporate 
Planning Division. Although many of 
these initiatives were developed under the 
purview of the Methods and Standards 
Committee, the Agency has not had a 
focal point with specific responsibility to 
ensure their co-ordination and integration. 
Nor does it have such a focal point now. 


3.89 We noted that definitions and 
requirements relating to quality of 
statistics are not always consistent. For 
example, the term “quality” itself has 
taken on new dimensions over time, 
expanding from the more traditional 
meaning of accuracy to one that reflects 
an explicit user orientation. Today, quality 
is defined or described differently in 
various policies, guidelines and systems. 


3.90 We also noted that internal and 
external reporting requirements are not 
co-ordinated and sometimes overlap. 
Besides being required to disclose 
information on data quality and 
methodology to users of each of their 
products, program managers are currently 
required to update the Statistical Data 
Documentation System annually, prepare 
Program Reports every two years and 
provide input to the Agency’s annual 
Performance Report to Parliament. 
Because these different reporting 
requirements are not co-ordinated, each 
one leads to additional work and can 
generate new streams of quality-related 
information. This may contribute to the 
difficulties that program managers and the 
Agency itself face in satisfying all the 
reporting requirements adequately. 


Bo | The Agency’s Policy on 
Informing Users of Data Quality and 
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Methodology establishes expectations for 
managers that are stronger than those 
established by the Quality Guidelines. The 
former clearly sets out “minimum 
standards” for disclosure by program 
managers. The latter — a “collection of 
methods, procedures and practices that 
govern the pursuit of quality objectives” 
— includes no mandatory requirements. 
As already noted, the Quality Guidelines 
give program managers latitude to select 
and implement whatever practices they 
consider appropriate in their particular 
circumstances. 


3.92 The fact that standards for 
informing users are mandatory, while the 
guidelines for documenting quality are 
discretionary, may contribute to program 
managers overlooking or violating 
mandatory minimum requirements to 
inform users. Both our audit and the 
Agency’s self-assessments showed that 
many of the programs examined did not 
comply fully with the Policy on Informing 
Users of Data Quality and Methodology. 


3.93 We believe that if quality were 
defined more consistently, and if quality 
assessment, documentation and reporting 
were better integrated, the burden on 
program managers and the Agency could 
be reduced. The Integrated Meta Database 
(the planned successor to the Statistical 
Data Documentation System) could be 
used consistently to document, for each 
survey, key quality-related decisions and 
the rationales for them, as well as quality 
indicators (as the Agency’s Quality 
Guidelines already suggest). The Database 
could then become a central repository of 
information on quality to support 
consistent and effective internal reporting 
and external disclosure. 


3.94 We believe that co-ordinated 
quality-related policies, guidelines, 
systems and processes, and a more 
disciplined approach to documentation, 
would help provide more systematic 
information about quality to support 
management and reporting. A more 
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disciplined approach means better, not 
necessarily more, documentation. 


3.95 Statistics Canada should make 
its quality-related policies, guidelines 
and systems more coherent and cohesive 
by: 


e reconciling differences in 
definitions of quality; and 


¢ co-ordinating the requirements for 
internal documentation and external 
reporting. 


Agency’s response: Agreed. 


3.96 Statistics Canada should 
co-ordinate the development of the 
Integrated Meta Database with other 
quality-related initiatives and take steps 
to ensure the ongoing completeness and 
reliability of the Database. 


Agency’s response: Agreed. The 
integration of quality-related information 


was already recognized as one objective of — A more disciplined 
the development of the Integrated Meta 


Database, and is under way. approach to 


documentation would 
help provide more 
systematic information 
about quality to 
support management 
and reporting. 


3.97, Statistics Canada should assign 
to a corporate focal point the 
responsibility for promoting an 
integrated, consistent approach to 
developing and implementing 
quality-related initiatives throughout 
the Agency, including the assessment 
and reporting of quality. 


Agency’s response: We agree with the 
objective underlying this recommendation. 
However, we believe that the most 
effective arrangement is to have the 
maintenance of quality as a prime 
responsibility of every line manager. We 
would be concerned about introducing any 
organizational arrangement that suggests 
to program managers that “someone else” 
is looking after quality issues. We will 
consider this issue further. 
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Using a hand-held computer, 
with on-line interactive editing, 
to collect price information for 
the Consumer Price Index 

(see paragraph 3.99). 


Conclusion 


3.98 In the course of our audit we 
noted Statistics Canada’s commitment to 
producing high-quality statistics and 
continuing to improve quality. We noted, 
too, that the Agency is widely respected 
among its peers, and has an international 
reputation second to none for 
independence, innovation and quality. 
Indeed, many employees of other 
well-regarded statistical agencies whom 
we interviewed indicated that they were 
complimented that we would look to their 
agencies as benchmarks for Statistics 
Canada, when in fact they were striving to 
emulate it. 


3.99 Statistics Canada has in place a 
wide range of policies and processes to 
ensure the ongoing relevance of its 
programs, to build quality in through 
design, execution and the use of new 
technologies, and to maintain an 
environment that encourages a concern for 
quality throughout the organization. 
However, we found that its achievement 
of quality is not sufficiently assessed and 
reported either within or outside the 
Agency. 


3.100 — Its formal quality assessment 
mechanisms are not sufficiently 
co-ordinated and are not applied 
consistently. Program managers do not 
always comply with reporting 
requirements. As a result, these 
mechanisms do not provide systematic, 
transparent information about the 
adequacy of quality management systems 
and practices in the Agency’s statistical 
programs or on the quality they actually 
achieve. 


3.101 However, the Agency’s existing 
policies, guidelines and systems can 
become the basis for more effective 
quality assessment and reporting practices. 
Integrating them better and taking a more 
disciplined approach to documentation 
would improve the nature and extent of 
information available to support internal 
decision making and assurance, as well as 
external reporting to Parliament and the 
public on performance, and to users on 
data quality and methodology. 


3.102 The four self-assessments 
Statistics Canada carried out for this audit 
were well planned and executed. All 
reached positive conclusions about the 
adequacy of quality management in the 
four surveys. For three of the four (the 
Consumer Price Index, the Labour Force 
Survey, the Monthly Survey of 
Manufacturing), we concluded that the 
self-assessments provided reasonable 
assurance that quality management 
systems and practices are adequate. In our 
judgment, the evidence presented in the 
self-assessment of the Uniform Crime 
Reporting Survey could have led to a 
stronger conclusion about the weaknesses 
identified and the importance of 
recommended improvements. 


3.103 The quality of statistics figures 
prominently in Statistics Canada’s 
effectiveness and in its commitments to 
Parliament for results. We therefore 
expected to find quality-related 
performance information in its most 
recent Performance Report to Parliament, 
tabled in October 1998. Overall, we 
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concluded that the Report provided only 
limited information on the Agency’s 
performance with respect to the quality of 
the statistics that it produces. 


3.104 Because statistics have to be used 
in full awareness of their strengths and 
limitations, we assessed Statistics 
Canada’s policy and practices for 
informing users about data quality and 
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methodology. We concluded that the 
Agency’s Policy on Informing Users of 
Data Quality and Methodology is well 
structured and sets out clear expectations 
for program managers. However, the 
Agency’s practices in informing users are 
inconsistent, and users are not always 
appropriately informed about the strengths 
and limitations of the statistics. 
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% About the Audit 


Objectives 
Our audit objectives were to determine whether: 


e Statistics Canada systematically assesses the adequacy of quality management systems and practices in 
its statistical programs to ensure that it has the information needed to manage and report on quality; 


e Statistics Canada’s self-assessments provide reasonable assurance about the adequacy of quality 
management systems and practices in the four selected surveys; and 


e Statistics Canada appropriately informs users about data quality and the methodology used to develop the 
statistics. 


Scope and Approach 


We examined various mechanisms used by the Agency to assess quality in individual statistical programs and 
to report results externally. We audited the Agency’s self-assessments of four selected programs: the 
Consumer Price Index, the Labour Force Survey, the Monthly Survey of Manufacturing and the Uniform 
Crime Reporting Survey. We also reviewed the Agency’s policy and practices for informing users about data 
quality and methodology, and reporting to Parliament on performance. 


In addition to auditing Statistics Canada’s self-assessments to determine whether we could rely on their 
conclusions, we reviewed documents and interviewed Agency staff. We also interviewed key users of 
statistics in the federal and provincial governments and the private sector. In addition, we compared the 
Agency’s approach to managing the quality of statistics with practices in a number of respected statistical 
agencies in other countries — including Australia, the Netherlands, Sweden, the United Kingdom and the 
United States. 


Criteria 


e The adequacy of quality systems and practices in individual statistical programs should be assessed 
systematically. 


e Assessment findings should be used in managing quality and in reporting performance. 


e Self-assessments should be properly planned and executed. Findings and conclusions should be supported 
by sufficient appropriate evidence. 


e Self-assessments should cover the key principles and criteria reflected in the Agency’s quality policies 
and guidelines. 


e The Agency should have an effective policy on informing users about data quality and methodology to 
help ensure informed use of its statistical products. 


e Individual statistical programs should appropriately inform users about data quality and methodology. 
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Appendix A 


Approaches to Quality Assessment and Reporting in Selected 
Statistical Agencies 


United Kingdom’s Office for National Statistics 


The United Kingdom Office for National Statistics (ONS) has an internal review process through which quality 
practices of major business surveys are audited periodically. A member of the team from outside the ONS helps ensure 
that the review is objective. 


For household surveys, data collection activities are subject to market testing. As part of the process, quality targets 
are clearly specified and the achievements by contractors (internal and external) are monitored. The surveys equivalent 
to the Labour Force Survey and the Consumer Price Index were two examples shown to us. 


In addition, a separate internal audit process helps assure the quality of the systems that produce statistical outputs and 
identifies any general risks associated with them for corporate action. 


In its annual Compliance Plan (submitted to Parliament), the ONS reports on changes in the response burden on 
businesses, by individual survey and in total. In the same document, planned and actual response rates for each survey 
are reported, providing some indication of data quality. 


The ONS launched its StatBase in October 1998. It includes a meta-database in which users can find information about 
the data quality and methodology of each survey. This initiative, we understand, is based on Statistics Canada’s 
Statistical Data Documentation System (SDDS). 


Statistics Sweden 


In Statistics Sweden, each survey must complete an annual “quality declaration” based on a 24-item self-assessment 
checklist. The objective is to determine whether the quality of the survey has improved, deteriorated or remained the 
same, in each of four characteristics: content (relevance), accuracy, timeliness and coherence. Division management 
reviews the results of the self-assessments in its area of responsibility. 


The results of these self-assessments are summarized for the agency’s management. They are also included in the 
agency’s annual report to Parliament. The Swedish National Audit Office audits the report and certifies the reliability 
of information on quality. Statistics Sweden’s self-assessment procedure is currently being revised so as to complement 
it with continuous measurement of key process variables and specific quality indicators. 


Statistics Netherlands 


Statistics Netherlands adopted an overall quality program in 1996. One of the objectives was to introduce quality 
systems in all statistical departments. Provisional guidelines were issued in 1997. At the same time, a system of 
“statistical auditing” was set up. The aim of this system is to check how quality management in statistical departments 
is functioning and how the quality of statistical products and processes (including the procedures) can be improved. 
Another objective is to document best practices and to incorporate them into the guidelines for quality management 
systems. 


All audits are carried out by a pool of about 25 agency staff, drawn from various divisions and working part-time on 
statistical auditing. They receive training from a private consulting firm with experience in auditing and quality 
management. 
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Appendix B 


Conclusions of Statistics Canada’s Self-Assessments 


Except where we make specific comments, we concur with the following conclusions of the self-assessments. 


Consumer Price Index (CPI) 


Relevance. “The CPI has continued to maintain a high degree of relevance in the context of Canada’s ever-changing 
economy, in spite of adverse budgetary conditions and the accompanying major reduction in the sample. 


Accuracy. “The constraints imposed by budget reduction have led to sample reductions in the program. However, 
these reductions were implemented without compromising the accuracy of the most widely used measures.” 


Timeliness. “The monthly measure of the CPI is produced in a timely fashion.” 
Accessibility. “Current information on the CPI is freely and widely available.” 


Interpretability. “The CPI program provides information on concepts and definitions and on issues of data quality 
through various channels, both through personal contact and through a series of publications designed to meet the 
needs of various users.” 


We agree with this statement. However, we noted that quarterly publications do not consistently refer the reader to 
sources where such information can be found. 


. Coherence. “The conceptual framework and classifications used in the CPI reflect the needs of its clients.” 


Labour Force Survey (LFS) 


Relevance. “The LFS program has adequate measures to assure relevance, including use of advisory committees, 
direct client-stakeholder feedback, and careful monitoring and response to media coverage. The program incorporates 
this feedback through decennial redesign, ongoing quality assurance committees, and a committee to manage major 
developments between decennial redesigns.” 


Accuracy. “The LFS is exemplary in terms of regularly monitoring the accuracy of data, primarily through the Data 
Quality Committee, which meets each month prior to the release of the survey results. A whole range of quality 
measures is monitored, including coefficients of variation, non-response, slippage, and coding error rates. Information 
products are reviewed in compliance with bureau policy.” 


Regional rates of unemployment are used, pursuant to the Employment Insurance Act, to determine the number of 
hours of work necessary to qualify for employment insurance benefits and the number of weeks of benefit. The LFS 
allocates its sample to achieve accuracy targets (coefficients of variation — CVs) for statistics on the unemployment 
rate in each Employment Insurance Region. The target CV, as established by a long-standing agreement between the 
Agency and Human Resources Development Canada (HRDC), is 15 percent. The Agency told us that occasionally the 
accuracy targets are not attained. When this happens, the Agency, in agreement with HRDC, may reallocate sample 
sizes to achieve the desired targets. 


Our review shows that the LFS quality reports did not include the CVs for the unemployment rates in each region. The 
Agency informed us that in the future these accuracy measures will be formally reviewed in the LFS quality reports, 
and that HRDC, the user of the information, will be regularly informed. 


Timeliness. “...the LFS results are released in very timely fashion — two weeks after the end of the survey collection 
period.” 
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Accessibility. “Information on program outputs is widely available to users and the public through Statistics Canada’s 
Daily and the LFS release on the Statistics Canada’s world-wide web site, through copies of the publication available 
free of charge at Regional Offices and depository libraries, and via wide media coverage of the highlights of the survey 
results.” 


We agree with this conclusion, but note that users consulted by the Agency have indicated some concerns about 
accessibility, including the cost of accessing labour market information. During our audit, several users we 
interviewed also expressed concerns about the high cost of LFS products. 


Interpretability. “LFS products conform to the Policy on Informing Users of Data Quality and Methodology.” 


The review team told us that although individual products may not conform to the requirements of the Policy on 
Informing Users of Data Quality and Methodology, collectively they are in compliance because they make reference to 
other documents. With the December 1998 release of a publication on the methodology of the LFS, users now have 
access to a wide range of quality indicators (for example, vacancy rates, non-response rates, design effects, sample 
sizes, sampling errors) 


Coherence. “The LFS has taken adequate measures to ensure coherence, including use of international standards for 
definition of key labour market variables, such as unemployment, employment and the unemployment rate. The 
program also uses standard classification systems for industry, occupation and geography. To further ensure coherence, 
the LFS program undertakes, in concert with other program areas, analysis of LFS estimates with those from other 
sources. 


We concur generally with the conclusion about coherence. However, two major users told us that the Agency could do 
more to improve the continuity of the LFS time series when technical changes, such as changes to classification 
systems, are introduced. 


There have been questions about the international comparability of unemployment rates. Although most Western 
countries follow the International Labour Organization (ILO) guidelines in measuring unemployment, each country 
may build into its surveys special features for its own needs. For example, while both countries follow the ILO 
guidelines, Canada includes “passive job seekers” in the unemployed but the United States does not. It is important to 
take account of such measurement differences when comparing unemployment rates. 


A discussion of this definitional difference and the resulting unemployment gap between the two countries was featured 
in the November 1998 issue of the Labour Force Update. LFS management told us that, in the future, key LFS 
publications would warn users of the dangers of international comparison of unemployment rates. 


Monthly Survey of Manufacturing (MSM) 


Relevance. “There is no evidence that the absence of a systematic, on-going external mechanism to ensure relevance 
is adversely affecting the MSM.” Nevertheless, the review team notes that “the MSM lacks a systematic 
communications link with its client community” and suggests a “more formal dialogue mechanism.” 


Accuracy. “It is the judgement of the [review team] that, given its budget, the accuracy aspect of quality is very well 
managed in this survey.” The self-assessment goes on to state, “The survey produces estimates of high standard as 
judged by measures of quality that are observed...response rates are usually in the nineties which compares favorably 
to the best sample surveys.” 


Discussions with MSM staff confirmed that with the stratified sample used in this survey, a small number of data 
sources represent a very large proportion of the value of the measured variables. In these circumstances, a more useful 
indicator of data quality than response rate might be, for example, the coverage achieved in terms of the value of 
shipments. 


Timeliness. “It is the judgement of the [review team] that the MSM’s timeliness is acceptable and, while there should 
be a continual striving to improve it, it should not be at the expense of a deterioration in accuracy.” 


We have no reason to question the judgment of the review team in this regard. However, we note that the report 
provides no evidence to support conclusions about timeliness versus accuracy. 
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Statistics Canada — Managing the Quality of Statistics 


Accessibility. “...the MSM information is readily available through Statistics Canada’s Daily publication, in printed 
copy and through electronic media.” 


Interpretability. “The survey largely complies with the agency’s Policy on Informing Users of Data Quality and 
Methodology.” 


Despite this conclusion, the review team recommends the inclusion of a new “Concepts and Methods” section in the 
key publication that “more clearly explains the data quality issues” and that includes tables and graphs to “illustrate 
the size and impact of revisions”. We agree with this recommendation. 


Coherence. “In the opinion of the [review team] the MSM does as good a job on the coherence dimension of quality 
as can reasonably be expected from a monthly survey.” 


Uniform Crime Reporting Survey (UCR). 


Relevance. “The UCR program has in place a superior means for both client and respondent liaison, consultation and 
feedback to ensure the relevance of its data and products.” 


The review team goes on to recommend, “The establishment of an advisory committee on analytical studies and 
requirements be considered to enhance the perception of independence of the UCR and to broaden the scope and 
extent of data use.” We agree with this recommendation. 


Accuracy. “The data are at most as good as the information available within the policing system itself...effective 
monitoring and assessment of quality needs an examination of data at the level of police forces. This is done in part 
through the editing process. Some of the primary checks and analysis at this level are performed by the respondents 
themselves, who “sign off’ on the results. (But this process) does not necessarily assure accuracy and it does not 
permit an independent assessment of accuracy. It does not account for differences in reporting and enforcement 
practices.” 


In 1997 Statistics Canada distributed a self-audit manual to police forces across the country. This initiative was based 
on the recommendation of a 1989 study, which found that police forces needed to conduct periodic audits of their 
internal systems to ensure accurate reporting of crime statistics. The Agency's follow-up with a number of major police 
forces showed that the forces did not have the resources to conduct the audits. In the meantime, there is continuing 
uncertainty about the quality of the information that is available within the policing system and reported to Statistics 
Canada. We were told that an internal proposal has recently been submitted within the Agency, requesting resources to 
perform a data quality audit with selected major police forces. 


Timeliness. “Given the nature of the survey, there is strong support for concluding that the data are as timely as is 
reasonable to expect under the current operational constraints...Any significant improvement in timeliness would have 
to come in respondent related activities.” 


Accessibility. “The practices used to disseminate and ensure accessibility of UCR data are consistent with Agency 
practices in general, and with requirements and views of major clients.” 


Interpretability. “In all cases (including the releases in the Daily), information on the concepts and definitions is 
given and amply meets the requirements and the intent of the Policy on Informing Users of Data Quality and 
Methodology...” 


With respect to information on data quality (as distinct from information on concepts and definitions), the review team 
makes three recommendations for improvement. 1) “An assessment of the impact of the coverage limitations of the 
UCR2 might be helpful (for example, by examination of aggregate data differences between the UCR1I and UCR2 
populations). 2) Cross-references in all products to the report “Canadian Crime Statistics” ... for details on 
methodology and data quality would also be useful ... 3) Response rates and an assessment of the effect of imputation 
by major variable should be provided.” Jn view of the nature of this survey, we agree with these recommendations and 
believe them to be important. 


Coherence. “There are clear and convincing attempts to provide a broad picture of criminal incidents, victims and 
perpetrators - integration of aggregate data across the surveys within the Police Services Program, integration of data 
from a variety of statistical sources (including some international data) and development across statistical programs. 
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Fisheries and Oceans 


Managing Atlantic Shellfish in a 
sustainable Manner 


The audit work reported in this chapter was conducted in accordance with the legislative 
mandate, policies and practices of the Office of the Auditor General. These policies and 
practices embrace the standards recommended by the Public Sector Accounting and 
Auditing Board (PSAAB) of the Canadian Institute of Chartered Accountants. The 
numbered paragraphs in bold face represent recommendations. 
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Main Points 


4.1 In October 1997, we reported on problems associated with the Department’s management of the Atlantic 
groundfish fisheries. In the current audit, we found that many of these problems also exist in the Department’s 
management of the Atlantic shellfish fisheries. For example, we noted increases in harvesting capacity and the 
encouragement of increased fisher participation through open access licensing in the shellfish fisheries. In 
addition, we found weaknesses in the information used in making resource decisions, and gaps in monitoring, 
control and surveillance. The full impact of these problems is not obvious, as most shellfish fisheries are currently 
recording high landed values. However, in our view these are significant concerns that must be addressed to 
ensure that the shellfish fisheries are managed in a sustainable manner. 


4.2 The Department’s decisions have a profound impact on those engaged in the fishing industry and the 
communities that rely on the income generated from the industry. The absence of a fisheries policy that fully 
reflects sustainability concepts means that decisions are made on an ad hoc and inconsistent basis rather than as 
part of an overall framework for achieving a sustainable fishery. An open and transparent process in which clearly 
articulated and consistently applied principles guide decision making would provide all stakeholders with 
assurance that their interests are considered and that the resource is protected over the long term. 


4.3 We observed resource use decisions that are not consistent with the Department’s currently stated 
objectives for fisheries management. As we reported in October 1997, there is a need to have the government 
clarify fisheries objectives in legislation. The Department needs to move forward with the development of a 
sustainable fisheries framework that incorporates the interdependent factors — biological, economic and social — 
that affect the fishery. 


Background and other observations 


4.4 In the 1990s, Atlantic Canada saw a virtual collapse of its commercial groundfish fishery (cod, haddock, 
pollock, halibut and various flatfish). In the same period, however, there was a general rise in the value of 
shellfish landings (lobster, scallop, snow crab and shrimp). In 1997, the landed value of all shellfish in Atlantic 
Canada was $920 million, which represented 81 percent of the landed value of all fish landed in the region. 


4.5 The Department has stated in its key parliamentary accountability documents that its objective is 
conservation, or protecting the productive capacity of the natural resource that supports the fishery. It has also 
reported that it has an economic objective, but the expected results for this objective are not stated. The 
Department has indicated that it is not responsible or accountable for social outcomes. We found that most 
resource use decisions in the shellfish fisheries are heavily influenced by social and economic factors. 


4.6 The Department’s “Fishery of the Future” strategy reflects objectives that include ensuring economically 
viable and self-reliant fisheries, over time. However, these objectives are not fully reflected in the Department's 
reporting to Parliament. We found resource use decisions in the shellfish fisheries that are inconsistent with the 
concept of an economically viable industry. 


4.7 Co-management, designed to increase industry’s role, responsibility and accountability in fisheries 
management, is an important aspect of the Department’s Fishery of the Future strategy. Participants assuming 
greater responsibility for their industry is an important element of sustainability. However, very little power 
sharing has actually occurred. In our opinion, there are weaknesses in the Department’s current approach to 
co-management. 


J) 
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4.8 The Department has recognized that there are weaknesses in the fisheries management framework in the 
existing Fisheries Act. However, amendments to deal with these weaknesses have not yet been re-introduced in 
the House of Commons. 


Fisheries and Oceans’ responses to our recommendations are included in this chapter. The Department 
either agrees to take action or notes that initiatives are under way to address three of our four 
recommendations. The Department has not indicated an intention to take action at this time on our 
recommendation that addresses co-management. 
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Introduction 


The rise in importance of shellfish 


4.9 In the 1990s, Atlantic Canada 
saw a virtual collapse of its commercial 
groundfish fishery. In the same period, 
however, the region’s commercial 
shellfish fisheries showed a general rise in 
the value of the catch. 


4.10 In fact, shellfish landings from 
1984 to 1997 rose in weight by 

138 percent — from 131,000 to 

312,000 tonnes. Since the early 1990s, 
landings have declined for lobster and 
scallops, but have significantly increased 
for snow crab and shrimp, primarily in 
Newfoundland and Labrador. 


4.11 At the same time, the rising 
demand for shellfish increased the prices 
received by fishers. Declines in the 
groundfish fishery notwithstanding, the 
landed value of all Atlantic Canadian fish 
reached $1.34 billion in 1995. When 
adjusted for the impact of inflation, this 
represented the second-highest annual 
landed value ever in the region. 

Exhibit 4.1 illustrates the trends in 
Atlantic landings and prices for the four 
major commercial shellfish species. 


The changing nature of the fishery and 
its impact on people 


4.12 Our October 1997 Report 
Chapter 14 discussed the dramatic impacts 
on Atlantic Canada following the collapse 
of most of its commercial groundfish 
fisheries. The labour-intensive nature of 
the fishery meant that its collapse affected 
large numbers of people in both the 
harvesting and the processing sectors. 


4.13 During the 1990s, there has been 
an unprecedented increase in incomes of 
most fishers involved in the shellfish 
sector. While generally providing good 
returns to fishers, historically this sector, 
with the exception of the lobster fishery, 
has not supported a large number of 


people. Nor does the shellfish sector 
require the same level of processing as 
groundfish, so it does not support as many 
plant workers. 


4.14 The incomes of fishers in the 
shellfish sector have provided a stark 
contrast to those of fishers who depended 
on groundfish for a living. The 1990s saw 
a variety of government programs 
designed to provide groundfish fishers 
with income support and opportunities for 
other employment, and they have become 
dependent in large part on these programs 
for a substantial portion of their income. 
In the end, the programs will have cost 
Canadian taxpayers over $3 billion. 


Focus of the audit 


4.15 This chapter reports on the 
management of major Atlantic shellfish 
fisheries by Fisheries and Oceans. We also 
report on the Department’s progress 
toward developing and implementing a 
management framework for sustainable 
fisheries, especially shellfish fisheries. 
The chapter assesses whether the 
Department has managed the shellfish 
fisheries in a way that contributes to 
meeting its own stated objectives for 
fisheries management. 


4.16 Our audit took a case approach. 
That is, we selected individual fisheries to 
determine whether and to what extent the 
objectives for fisheries management were 
pursued consistently. 


4.17 The cases selected for audit were 
in the lobster, scallop, snow crab and 
shrimp fisheries. In 1997 these fisheries 
represented 92 percent of the value of 
shellfish landed in Atlantic Canada; 
shellfish in total represented 81 percent of 
the value of all fish landings in the region. 
Exhibit 4.2 shows the specific areas in 
these four fisheries that we included in our 
audit. We selected the fisheries that not 
only have the highest value but also 
include all of the major fishing areas of 
Atlantic Canada. 


In 1997 shellfish 
represented 

81 percent of the 
value of all landings 
in Atlantic Canada. 
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Exhibit 4.1 
Trends in Landings and Prices for Lobster, Scallop, 
. : Landings Price 
Snow Crab and Shrimp - Atlantic Canada 
LOBSTER SCALLOP 
Metric Tonnes Average Dollars Metric Tonnes of Meats Average Dollars 
(in thousands) per Pound (in thousands) per Pound 
50 5.00 16 7.00 
14 6.00 
40 4.00 
12 
5.00 
4 10 
30 — 3.00 4.00 
8 
3.00 
20 2.00 6 
Ph. 
4 00 
10 1.00 
2 1.00 
, ‘ i : : 
1972 1976 1980 1984 1988 1992 1996 1972 1976 1980 1984 1988 1992 1996 
SNOW CRAB SHRIMP 
Metric Tonnes Average Dollars Metric Tonnes Average Dollars 
(in thousands) per Pound (in thousands) per Pound 
80 3.50 80 1.40 
70 3.00 70 1.20 
60 
: 2.50 " 1.00 
50 50 
2.00 0.80 
40 40 
1.50 0.60 
30 30 
20 1.00 20 0.40 
10 0.50 10 0.20 
) 0 0 
1972 1976 1980 1984 1988 1992 1996 1972 1976 1980 1984. 1988 1992 1996 


Source: Fisheries and Oceans Canada 
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4.18 Further details are provided in 
About the Audit at the end of the chapter. 


Observations and 
Recommendations 


Fisheries Management Framework 


The powers of the Minister of Fisheries 
and Oceans 


4.19 The Fisheries Act provides 
extensive powers to the Minister of 
Fisheries and Oceans. For the fisheries 
within the jurisdiction of the federal 
government, the Minister has the power to 
decide who fishes (by issuing fishing 
licences), how much fish can be harvested 
(by allocating quotas), the fishing methods 
to be used, the timing of the fishing 
season and many other aspects of fishing 
activity. 


The responsibilities of the Department 
of Fisheries and Oceans 


4.20 In its 1998-99 Part III Estimates 
(Report on Plans and Priorities), the 
Department stated that the overall 
departmental objectives include, among 
other things, undertaking policies and 
programs in support of Canada’s 
economic, ecological and scientific 
interests in the oceans and inland waters 
and providing for the conservation, 
development and sustainable economic 
utilization of Canada’s fisheries resources 
for those who derive their livelihood or 
benefit from these resources. The specific 
objective of fisheries management is: 


...[to ensure] the conservation and 
protection of Canada’s fishery 
resource and, in partnership with 
stakeholders, to assure its sustainable 
utilization. 


In its key accountability documents, the 
Department has informed Parliament that 
its objective for fisheries management is 
“conservation”. At the beginning of our 
audit, the Department informed us that 
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“conservation” as used here means the 
biological aspects of sustainability; it does 
not include broader economic or social 
issues. In the 1998—99 Report on Plans 
and Priorities, none of the expected results 
for fisheries management refer to the 
Department’s overall economic objectives. 


Defining fisheries management 
objectives 


4.21 Since Program Review in 1994, 
the Department has been pursuing the 
“Fishery of the Future” strategy, to make 


fisheries: The Minister has 


e economically and environmentally extensive powers. 


sustainable; 


e stable and capable of providing 
adequate levels of income; and 


e self-reliant, competitive and viable 
without subsidization. 


These objectives are not fully reflected in 
the Department’s 1998-99 Report on 
Plans and Priorities, although the 
document refers to the Fishery of the 
Puture strategy. 


4.22 Moreover, neither the 
Department’s reports to Parliament nor its 
Fishery of the Future strategy reflect any 
consideration of the social implications of 
fishery management decisions. In a 

26 August 1998 response to the Standing 


Exhibit 4.2 


Shellfish Stocks Included in Our Audit 


Stocks Fishing Areas | 
Lobster e Southwest Nova Scotia 
e Southern Gulf of St. Lawrence | 
Scallop e Offshore Nova Scotia 
e Bay of Fundy 
Snow Crab e Newfoundland 
e Gulf of St. Lawrence (except for Laurentian 
inshore areas) 
Shrimp e Northern shrimp 
e Gulf of St. Lawrence 
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For certain decisions, 
the Department has 
not adhered to its 
Fishery of the Future 
strategy. 


Committee on Public Accounts, the 
Department confirmed that it does not 
support consideration of social factors : 


A reversal of current fisheries 
management policy principles and a 
return to a social fishery to 
accommodate former TAGS (The 
Atlantic Groundfish Strategy) 
recipients would result in an increased 
conservation risk, decreased industry 
viability and greater costs to 
government in the long run in the 
form of higher management costs and 
greater access to EI (Employment 
Insurance). 


4.23 However, 1n certain important 
decisions the Department has not adhered 
to the Fishery of the Future strategy, 
which represents its vision for the fishery. 
For example, Exhibit 4.3 describes three 
allocation decisions made to support 
social objectives that are not reflected in 
the strategy. 


4.24 Another example of the 
Department’s pursuit of social objectives 
is in the Northern shrimp and Gulf snow 
crab fisheries. In 1997 and (for Northern 
shrimp only) in 1998, temporary 
allocations were approved for fishers’ 


organizations and community-based 
organizations, either directly or to 
companies controlled by them. The 
purpose of these temporary allocations 
was to strengthen or support initiatives by 
the organizations and to redistribute 
income from these lucrative fisheries to 
the organizations’ members. We estimate 
that the total landed value of these 
temporary allocations was approximately 
$15 million. 


4.25 We found that prominent themes 
in the management of certain shellfish 
fisheries have been the distribution of 
wealth and the sharing of benefits between 
the fishers pursuing “successful” fisheries 
and other licensed fishers, including those 
dependent on groundfish. In other 
lucrative shellfish fisheries, however, no 
attempt has been made to engineer a 
sharing of the fishery’s wealth. 


4.26 The Fisheries Act provides the 
Minister and the Department with the dual 
roles of protecting the productive capacity 
of the natural resources that support 
fisheries and allocating the resource to 
those licensed to fish. In its reporting to 
Parliament, to the media and to the 
general public, the Department 


Exhibit 4.3 e The 1997 inshore allocation in shrimp fishing area 5, intended to be fished by inshore vessels 
under 65 feet, was allocated to several community-based Labrador corporations. These 
corporations, in turn, chartered or sold the allocations to the existing offshore licence holders, 
who in some cases made further chartering arrangements. The same practice occurred in 1998. 


Three Resource Allocation 
Decisions Reflecting Social 


Objectives e In 1997 and 1998, an allocation of 3,000 tonnes of shrimp in shrimp fishing area 6 was granted 


to a community-based corporation that had the goal of developing a shrimp processing facility 
on the Northern Peninsula of Newfoundland. In both years, the corporation contracted with 
others to fish and process the allocation. 


e Fisheries and Oceans defines an exploratory fishery as one where it is attempting to determine 
whether a stock can sustain a commercially viable operation, and to collect biological data for 
information on stock abundance and distribution. In 1995, the Minister authorized the 
establishment of two exploratory snow crab fishing zones in the Gulf of St. Lawrence area, to be 
fished by groundfish-dependent fishers who did not hold snow crab licences. This fishery was 
established as part of a strategy to “meet the demands for more equitable sharing of the snow 
crab resource in areas where it continued to be abundant”. In 1997, the Department reported to 
the Minister that these two zones no longer met the requirements for an exploratory fishery, as 
scientists had concluded that the zones had no resident stocks. The snow crab in the zones was, 
in fact, migrating from the adjacent snow crab fishing area 12. The Department recommended 
that the exploratory fishery in these zones be closed. The Minister chose to keep the zones open 
in 1998. While this fishery was approved as an “exploratory” fishery, fishers have never been 
required to complete the normal scientific protocols expected in such fisheries. 
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emphasizes the first role, protection. In 
practice, the day-to-day activities of 
fisheries managers deal for the most part 
with the second role, allocating the 
resource. The result of the Minister’s 
allocation decisions is the distribution of 
wealth from the fisheries. 


4.27 There is a natural tension 
between the roles of protecting resources 
for future use and allocating resources for 
current use. Social and economic 
pressures on the Department and the 
Minister will generally be to allocate 
resources for current use. These pressures 
are hardest to manage when the 
Department has limited knowledge to 
prove how much of the stock must be left 
unharvested to protect the resource. When 
knowledge about stocks is limited, it is 
particularly important that the 
precautionary approach be applied — that 
is, erring on the side of caution. 


Policies governing the fishery are not 
always clear 


4.28 Resource allocation and licensing 
decisions have important economic and 
social consequences. The Fisheries Act 
gives the Minister of Fisheries and Oceans 
absolute discretion to make such 
decisions. Historically, in making these 
decisions, the Minister and the 
Department in support of the Minister 
have considered a number of factors, 
including those of a social and economic 
nature. Examples of these factors include 
fishers’ adjacency to the resource, 
historical attachment to the stock and 
historical provincial share. In December 
1995, the Department adopted the 
Commercial Fisheries Licensing Policy 
for Eastern Canada (1996), which set out 
its revised position on issuing licences. 
This Policy makes reference to a number 
of factors, including those mentioned 
above, but it does not clearly establish 
how these factors are to be reflected in 
practice. The Policy allows for exceptions 
based on geographic and economic 
circumstances, which are not specified. 


4.29 The difficulties of applying this 
Policy in practice can be demonstrated by 
the following examples: 


e The Gulf shrimp fishery has 
historically been fished by Quebec, New 
Brunswick and Newfoundland fishers. In 
1998, the Minister approved the allocation 
of 150 tonnes of Gulf shrimp to Nova 
Scotia and Prince Edward Island fishers. 
This temporary arrangement alters the 


historical provincial share of this stock. There is a natural 


tension between 
protecting resources 
for future use and 
allocating resources 
for current use. 


e The union representing crews on 
offshore scallop vessels requested that the 
Minister grant it a percentage of the 
offshore scallop quota. In an internal 
review, the Department indicated that the 
request for the allocation could not be 
granted independent of a licence, meaning 
that a new offshore scallop licence would 
have to be issued to the union. This, in 
essence, would mean abandoning or 
providing an exception to the limited 
entry policy that had been in effect for the 
offshore scallop fishery since 1973. The 
allocation has not been granted to the 
union. In contrast to this position, 
however, Gulf snow crab with an 
estimated landed value of $5 million was 
allocated in 1997 to companies set up and 
controlled by fishers’ organizations, 
including unions. These companies were 
issued new licences to fish snow crab. 
Moreover, in the Northern shrimp fishery 
the Minister allocated 2,000 tonnes of 
shrimp to the union representing 
Newfoundland fishers and to a fishers’ 
co-operative, even though they were not 
licence holders. This allocation was fished 
through arrangements with existing 
offshore licence holders. 


4.30 A recent review conducted on 
behalf of the Department’s Review 
Directorate observed that there is a 
“two-track” approach to resource 
allocation. The first track is based on 
Integrated Fisheries Management Plans, 
which departmental officials and the 
industry develop together to determine 
how the fishery should be managed. The 
second track is outside that process: 
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Exhibit 4.4 


A “Two-Track” Allocation 
Approach — Northern Shrimp 


industry groups or others address their 
concerns directly to the Minister either 
before the formal process or concurrently 
with it. We noted the latter approach in 
the allocation decision for the Northern 
shrimp fishery (see Exhibit 4.4). We 
observed that the same approach was also 
used in the consultations on changes to 
harvesting practices for Gulf lobster. 
While we understand that the Minister has 
absolute discretion in this area, we are 
concerned that the absence of 
transparency in fisheries management 
decisions has the potential to undermine a 
long-term perspective on sustainability of 
the resource. 


What does the Department mean by an 
economically viable fishery? 


4.31 The Department’s Fishery of the 
Future strategy calls for an industry that is 
economically viable, over time. In 
general, economic viability means that 
those engaged in an activity receive 
sufficient returns to stay in business and 
enjoy a “reasonable” living without 
subsidies from government. The 
Department has not elaborated on what it 
intends to achieve when it calls for an 
industry that is “economically viable, over 
time”’. 


4.32 We observed resource allocation 
and licensing decisions that did not appear 
to be consistent with the objective of 
economic viability. For example: 


e As little as one tonne of snow crab 
was allocated to inshore Newfoundland 


fishers. In this instance, the apparent goal 
was to maximize employment at marginal 
incomes rather than to achieve economic 
viability. The case study on page 4-13 
describes the impact of management 
decisions on the economic viability of the 
Newfoundland snow crab fishery. 


e Inthe Northern shrimp fishery, the 
Department’s actions have encouraged 
increased harvesting capacity in the 
industry even though there is uncertainty 
about how long the recent increases in this 
stock will last (see the case study on 
page 4-14). 


The role of government in controlling 
industry’s harvesting capacity 


4.33 The Food and Agriculture 
Organization of the United Nations has 
reported that a common problem in 
fisheries around the world is overcapacity, 
as fishers invest in larger or more efficient 
vessels and equipment. When they see 
that overcapacity has become a serious 
problem, governments can do one of three 
things. They can recognize the risk to the 
stocks and provide financial incentives to 
reduce capacity; this has been a favoured 
approach in Canada, although often only 
after major stock declines, as happened in 
the Atlantic groundfish and Pacific salmon 
fisheries. They can allow overcapacity to 
go unchecked, which then puts a strain on 
the resource. Or they can encourage 
industry self-rationalization through 
management regimes such as enterprise 
allocations or individual transferable 
quotas. In all cases, people who rely on 
the resource usually look to government 


In 1996, the Minister sought industry views and proposals on how to share increases in Northern 
shrimp quotas for 1997 and beyond. The results of the formal consultative process with industry 
were presented to the Minister on 11 April 1997 in a memorandum from the Deputy Minister. The 
Minister was asked to approve a formula for sharing of the increased quotas between the existing 
offshore licence holders and potential new entrants. This proposal was not accepted by the 
Minister. The Minister ultimately accepted a proposal put forward by his staff, the union 
representing the potential new entrants and the Department’s Newfoundland Region. This new 
proposal recommended that all of the increase in quotas be given to new entrants and selected 
offshore licence holders, based on the adjacency principle. This proposal’s definition of 
adjacency to the resource resulted in almost exclusive access by Newfoundland inshore fishers to 
the substantially increased quota in 1997 and 1998. 
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for assistance until the resource recovers, 
or until it becomes clear that the resource 


will not recover. A recent example was 


the income support program for those who 


were dependent on Atlantic groundfish 
stocks. 


4.34 Since 1992 the Department has 
made efforts through licensing policy 


initiatives and expenditures to reduce the 


harvesting capacity of the groundfish 
sector. The Department has spent 
approximately $85 million to eliminate 
over 1,300 groundfish licences. The vast 
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majority of the licences retired were in the 
Newfoundland Region. Nevertheless, as 


noted in our October 1997 Report 
Chapter 15, excess harvesting capacity 


continues to be a concern. 


4.35 


Since 1997 the harvesting 


capacity in the Newfoundland Region has 
increased dramatically. The increase has 
come in the form of 208 new shrimp 
trawlers (see the case study on page 4-14) 
and 150 vessels enlarged or replaced 
under Supplementary Vessel Replacement 
Rules for the purpose of fishing snow 


Newfoundland Snow Crab Fishery — Effect of Open Access on Economic Viability 


In recent years, the number of individuals 
in the Newfoundland snow crab fishery has 
significantly increased. Prior to 1995, this 
fishery was conducted by 822 licensed fishers 
(see chart). In 1997, the Minister opened the 
snow crab fishery to all Newfoundland core 
fishers through the issuance of “temporary” 
licences. There are now 3,182 licensed snow 
crab fishers. The new temporary fishers are 
predominantly from the small boat inshore 
fishing fleet, with vessels less than 35 feet. This 
group was particularly affected by the closure of 
the groundfish fishery. 


Large numbers of the new entrants have 
been granted very small quotas that produced 
low income per individual. In 1997, individual 
quotas for the temporary inshore fishers were 
as low as 1 tonne, providing about $2,000 in 
revenue, and averaged 2.7 tonnes, providing 
about $5,400 in revenue. Although the Minister 
originally approved a minimum quota of 
3 tonnes per individual to ensure a minimum 
level of economic viability for new entrants, he 
subsequently reduced the minimum quota to 
1.8 tonnes. This change was made after 
industry made representations that open access 
be allowed. The Department further lowered 
the minimum to 1 tonne in order to 
accommodate all fishers. Departmental data on 
landings indicate that the new entrants rely on 
this fishery as a primary source of fisheries 
income. 


The traditional snow crab fishers have 
been required to vacate inshore zones to 
accommodate the temporary entrants. They 
must now fish in areas ranging from 50 to 200 
miles offshore. This displacement has increased 
their operating costs. So they would accept the 


displacement, the Department provided the 
traditional fishers with additional quota to offset 
their increased operating costs. In some of 
these new areas, there is uncertainty about 
stock status and the long-term viability of the 
snow crab resource. 


In 1998, the total allowable catch was 
increased by 9.5 percent above the 1997 
level to accommodate the temporary inshore 
fishers. The increase was contrary to the 
two-year management plan, which committed 
to no increases in quota unless substantial 
changes in stock status occurred. The 
Department's Science Branch had initially 
indicated that there was no basis for 
suggesting a change to the management 
plan for 1998. In 1998, no formal Stock 
Status Report was produced for 
Newfoundland snow crab. 


Our concerns. We recognize the 
pressures under which the Department must 
operate, especially in Newfoundland where 
chronic unemployment is a societal challenge. 
However, we are concerned that open access 
to the Newfoundland snow crab fishery has led 
to the entry of large numbers of fishers who are 
receiving marginal incomes at a time when this 
fishery is at historically high levels. The 
management of this fishery demonstrates the 
Department's difficulty in balancing competing 
biological, social and economic factors. 
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crab. Approximately 50 of the 150 vessels 
were under 35 feet long and have been 
replaced with vessels between 35 and 45 


feet. 


4.36 
define and determine the harvesting 


capacity of industry, let alone control it. 
For example, the new and the enlarged 
vessels have a large potential harvesting 


capacity, which could be readily 


It is difficult for government to 


redirected to harvest groundfish. We are 
concerned that this increase in potential 
harvesting capacity may offset the 
government’s previous efforts to reduce 
capacity in the groundfish fishery. The 
Department has indicated that it has the 
means to control the transfer of capacity 
from the shellfish to the groundfish 
fishery. However, past experience has 
shown that government comes under 


Northern Shrimp Fishery - Encouraging Growth in Harvesting Capacity Without Long-Term Knowledge 


In the past two years, the Northern 
shrimp fishery in the Newfoundland Region has 
grown significantly (see chart). 


Science advice. In the 1997 Stock Status 
Report for Northern shrimp, the Science Branch 
concluded that the stock was at an 
unprecedented level and was widely distributed. 
In 1998, advice provided from outside the 
formal stock status process indicated that the 
current exploitation level is low and does not 
immediately jeopardize the stock. However, the 
Science Branch also noted that this current high 
abundance might be a pulse, and that future 
rapid declines in biomass can be expected. 
Furthermore, the Science Branch could provide 
assurances about the stock for only two to 
three years. As noted in paragraph 4.61 of the 
chapter, the Science Branch did not consider 
the doubling of the 1998 total allowable catch in 
shrimp fishing area 6 to be precautionary. 


The response of fisheries 
management to the stock increase. The total 
allowable catch for area 6 was doubled in 1997 
and again in 1998 for a total increase of 
318 percent since 1996 (from 11,050 tonnes to 
46,200 tonnes). Open access to this fishery 
was provided by the Minister to all 
Newfoundland core fishers through the 
issuance of temporary shrimp licences in 1997 
and 1998. A condition of these temporary 
licences was that each fisher was required to 
use his or her own vessel. As a consequence 
of this open access management regime, the 
number of participants increased significantly. 
For example, prior to 1997 there were 17 
licence holders fishing Northern shrimp in area 
6, with a total allowable catch of 11,050 tonnes. 
In 1998 there were an additional 300 licence 
holders in area 6 with an additional total 
allowable catch of 35,000 tonnes. 


Fleet capitalization. In response to the 
increased total allowable catch, the open 
access fisheries management regime, and the 
requirement for all participants to be equipped 
to harvest shrimp in order to gain access, 
industry invested heavily in this new fishery. 
Specifically, 208 of the new licencees have 
invested in new shrimp trawling gear. The 
Department estimates that the average cost of 
new trawler gear is $150,000 per vessel or a 
total gear-up cost of about $30 million since 
1997. Although the value of the shrimp 
landings by these new shrimp trawlers was 
about $26 million in 1998, the reliability of this 
revenue over time is not known given the 
scientific uncertainties. 


Capacity growth. Fleet capitalization has 
led to a significant increase in harvesting 
capacity in the Newfoundland Region. With 
minor modifications, the 208 new shrimp 
trawlers can also be used to harvest other 
species such as groundfish. Onshore 
processing has also significantly increased 
since 1996. 


Our concerns. The Department has 
encouraged significant growth in harvesting 
capacity while the expectations are that the size 
of the shrimp stock will decline rapidly at some 
point in the future. We are also concerned that 
the size of the trawler fleet in Newfoundland 
has increased significantly at a time when the 
government has taken action, through a series 
of licence retirement programs, to reduce 
overall harvesting capacity in Atlantic fisheries. 
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intense pressure to allocate stocks to meet 
capacity needs. 


4.37 We observed that harvesting 
capacity has also increased in the lobster 
fishery, due mainly to improved efficiency 
of fishing gear, larger and more powerful 
boats, and improved fishing technology. 
This was one of the main reasons behind 
the Minister’s recent plan for additional 
controls over harvesting in this fishery. 
Conversely, in the offshore and Bay of 
Fundy scallop fisheries, the Department 
has used enterprise allocations and 
individual transferable quotas respectively 
to enable licence holders to reduce 
capacity in line with availability of the 
resource. The number of vessels in the 
offshore scallop fishery was reduced from 
66 in 1985 to 32 in 1996. The fleet sector 
that harvests 85 percent of the Bay of 
Fundy scallops in the inshore only 
recently adopted a self-rationalizing 
system and it is too early to determine the 
results of this new approach. 


Departmental action to develop a 
fisheries management framework 


4.38 Our October 1997 Report 
Chapter 14 dealt with the management of 
the Atlantic groundfish fishery. We 
believe that the observations and 
recommendations we made then are 
relevant to the management of any fishery. 
We recommended that Fisheries and 
Oceans: 


e renew its efforts to have the 
government clarify fisheries objectives in 
legislation and develop a national fisheries 
policy framework; 


* pursue, as a priority, completion of a 
consolidated policy framework related to 
sustaining the fisheries resource base; and 


e establish measurable indicators and 
performance expectations to assess 
progress in applying guiding principles, 
and integrate those indicators in its 
planning, reporting and accountability 
process. 
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4.39 The Department responded that it 
would take action on our recommenda- 
tions by seeking amendments to the 
Fisheries Act, preparing a framework to 
elaborate on its policy objectives and 
principles, and developing performance 
measures and business plans. 


4.40 On 26 August 1998, the 
Department provided the Standing 
Committee on Public Accounts with 
updates and action plans for implementing 
our recommendations. On the 
development of a fisheries management 
framework the Department stated, among 
other things, the following: 


e Since Program Review, it had 
pursued two interdependent and 
complementary objectives: the Fishery of 
the Future strategy — industry 
restructuring aimed at producing a sector 


The Department has 
that is economically and environmentally 
sustainable and self-reliant; and indicated that it does 
management renewal — aimed at not have the 
refocussing the Department on its primary 
mandate of conservation and sustained use responsibility or 


of the resource. Concurrent with budget 
reductions, this would also mean 
devolving to fish harvesters a greater 
decision-making role and cost 
responsibility in resource conservation and 
management. 


resources to manage 
social and economic 
outcomes. 


e The Department would explore ways 
and means of changing the existing policy 
framework by December 1998 to be more 
consistent with the objectives of the 
Fishery of the Future strategy. 


e As the Department had not been 
assigned the responsibility or the 
resources for managing the social and 
economic outcomes, it would not be 
developing related performance measures. 
Specifically, it said: 


On the measure of balancing industry 
capacity to establish an economically 
and socially viable industry, it was 
viewed that the mandate of the 
Department was revised in the 
Program Review exercise to focus on 
the core business of conservation, and 
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that we have not been assigned either 
the responsibility or the resources to 
manage the socio-economic 
outcomes. As a consequence, at the 
“Outcome” level, fisheries 
management will not measure or 
target performance in this area. The 
only exception to this rule is in 
situations where the Government has 
conferred upon the Department 
special responsibilities. In these cases, 
socio-economic outcome performance 
indicators would be developed as 
appropriate. 


e The Department was committed to 
developing a performance measurement 
framework for fisheries management, to 
include updating the framework in the 
Multi-Year Strategic Plan. Measures that 
focussed on outcomes were to be 
established and implemented by 
December 1998. 


In our view, the Department’s position in 
this response to the Committee is 
inconsistent. The Department states that it 
has an objective of producing a sector that 
is “economically... sustainable and 
self-reliant”. Yet, at the same time, it says 
it does not have the “responsibility or 
resources to manage the socio-economic 
outcomes.” 


4.41 At 31 December 1998, the status 
of initiatives taken in response to our 
recommendations was as follows: 


e Amendments to the Fisheries Act. 
These have yet to be tabled in the House 
of Commons. In late 1998, the Minister 
established a Panel to examine the key 
issue of partnering arrangements — an 
issue that underlies a number of the 
proposed amendments. The Panel reported 
on 10 December 1998 and “urged the 
Minister of Fisheries and Oceans not to go 
forward at this stage with legislation for 
partnering”, largely due to widespread 
opposition to the concept. 


e Sustainable fisheries policy 
framework. In October 1998, 
departmental officials conducted a 


strategic planning workshop. Updated 
draft vision and mission statements were 
produced and objectives and strategies to 
achieve the updated mission were 
determined. Senior management proposed 
objectives that clarified the meaning of 
industry viability, the intent of fisheries 
management and the social dimensions of 
the Department’s responsibilities. As a 
result of this exercise, departmental 
officials have determined that social and 
economic considerations do have a role in 
fishery decisions. We recognize the 
Department’s efforts to clarify what it is 
trying to achieve through its fisheries 
management activities; yet these initial 
efforts need to be formally approved and 
incorporated into its key accountability 
documents. 


e Development of a performance 
measurement framework. The 
Department has deferred this work 
pending formal adoption of the fisheries 
policy framework. 


Sustainable fisheries framework 


4.42 A sustainable fisheries 
framework is the sum of the legislation, 
policies and principles that establish what 
society wants to achieve from the fishery; 
it also guides the conduct and 
management of the fishery over the long 
term. That is, having first determined 
what society is trying to achieve through 
the fishery, the Department should be able 
to establish the most cost-effective means 
of achieving it 1n a sustainable way. In this 
context, we have assumed a definition of 
“sustainable fishery” that is consistent 
with the definition of “sustainable 
development” in the Auditor General Act: 


“sustainable development” means 
development that meets the needs of 
the present without compromising the 
ability of future generations to meet 
their own needs. 


4.43 A sustainable fisheries 
framework takes into consideration the 
interdependent factors — biological, 
economic and social — that affect the 
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fishery. Thus, it is by nature complex, 
multi-dimensional and constantly 
evolving. In the Appendix, we report on 
the “lessons learned” from our review of 
fisheries frameworks in other jurisdictions 
and from the Atlantic Canada experience. 


Summary 


4.44 Sustainable fisheries require that 
interdependent biological, social and 
economic factors be considered from a 
long-term perspective. In addition to its 
“conservation” role, the Department has 
indicated that it is pursuing economic 
objectives for the fishery. However, the 
expected outcomes of pursuing this 
economic objective have not been 
identified. We have also noted that the 
Department is making decisions that 
support unspecified social objectives. The 
absence of a formal fisheries policy that 
fully reflects sustainability concepts 
means that decisions on resource use are 
made on an ad hoc and inconsistent basis 
rather than as part of an overall 
framework for achieving a sustainable 
fishery. 


4.45 The Department of Fisheries 
and Oceans (DFO) should move 
forward with the development of a 
sustainable fisheries framework, 
including efforts to have the 
government clarify fisheries objectives 
in legislation. The framework should 
guide the conduct and management of 
the fishery over the long term. 


Department’s response: DFO has recently 
established a Working Group on the 
Reform of the Atlantic Fisheries Policy, 
under the responsibility of the Assistant 
Deputy Minister Fisheries Management 
and Assistant Deputy Minister Policy. 
Part of the mandate of this group will be 
to deal with the creation of a cohesive and 
consistent policy framework providing a 
broad vision of the direction of future 
fisheries in the Atlantic. 
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Implementing “Conservation” in 
the Shellfish Fisheries 


What does “‘conservation”’ mean to the 
Department? 


4.46 As we have noted (paragraph 
4.40), since Program Review the 
Department has focussed on its core 
business of “conservation” of Canada’s 
fishery resource. How does the 
Department know when it has 
“conserved” fishery resources? Does it 
know what a healthy stock is? 
“Conservation” could have a number of 
possible objectives, such as increasing the 
overall size of the stock, maintaining the 
status quo or increasing the size of the 


stock available to spawn. Sustainable fisheries 


require the balancing 
of interdependent 
biological, social and 
economic factors. 


4.47 Each of the fisheries we 
examined had a planning process. All 
fisheries management plans that we 
examined included “conservation” as an 
objective, but only the objective for the 
lobster fishery was quantified. (We noted 
that the Department views the objective 
for the lobster fishery as an interim step 
while it develops better measures of 
“conservation”). Examples of objectives 
stated in the fisheries management plans 
included the following: 


e to double eggs per recruit over a 
two- to four-year period (Atlantic lobster); 


e toimplement management measures 
that will protect and conserve snow crab 
resources (Newfoundland snow crab); 


e to promote and ensure the 
conservation and protection of the snow 
crab resource (Gulf snow crab areas); 


e toensure the conservation and 
restoration of the resource (offshore 
scallops); 


e to make conservation of the resource 
paramount (Northern shrimp); and 


¢ to protect small scallops juveniles) 
from recruitment and growth overfishing 
(Bay of Fundy scallops). 


4-17 
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Most fisheries 
management plans 
did not include clear 
expectations of what 
is to be achieved. 


4-18 


4.48 Only the Atlantic lobster 
objective is clear and measurable. The 
other fisheries management plans include 
no clear expectations of what is to be 
achieved on either a short-term or a 
sustained basis. 


4.49 Having the right information to 
make decisions about resource use is 
important so the decisions do not 
endanger the health of the stocks. The 
Department has indicated that when the 
status of stocks is uncertain, decisions on 
resource use will take a precautionary 
approach — that is, decision makers will 
err on the side of caution. 


4.50 In addition to a lack of 
measurable goals and targets, we observed 
management decisions that did not reflect 
the “conservation” concerns of 
departmental officials. The case study on 
page 4-19 discusses a fisheries 
management decision to allow the harvest 
of scallops in an important nursery area 
for the Bay of Fundy scallop stocks, which 
are critically low. 


Stock assessment 


4.51 The Department’s Science 
Branch conducts annual stock assessments 
using information from research surveys 
and studies as well as information on the 
commercial fishery. It also consults with 
the industry and fisheries managers on the 
status of the stock. The Science Branch 
then provides advice to the Fisheries 
Management Branch in the form of Stock 
Status Reports. These reports are 
important input for decisions by the 
Minister, or in certain instances by 
fisheries managers, on how a fishery will 
be conducted, including the amount of 
fish that can be caught. 


4.52 In some of the fisheries we 
examined, we found weaknesses in the 
information available to support resource 
use decisions. For example: 


e Inall lobster fisheries, there is 
limited knowledge of the role that the 


various environmental and ecological 
factors play in determining reproductive 
success. While the Department has used 
one biological reference point to guide its 
resource use decisions, it has recognized 
the need to develop additional reference 
points to assist in its assessment of the 
status of the lobster stocks (the case study 
on page 4-20 describes the problems of 
gathering knowledge in the lobster 
fishery). 


e A total allowable catch for the Bay 
of Fundy scallop fishery was established 
for the first time in 1997. This fishery is 
divided into eight fishing zones. The 
Department has only limited scientific 
information to support catch limits in 
certain zones of this fishery. 


e The advice provided by the Regional 
Science Branch for both the 1998 
Newfoundland snow crab and Northern 
shrimp fisheries did not follow the usual 
departmental practice of preparing formal 
Stock Status Reports (see the case studies 
on page 4-13 and page 4-14 for further 
information). The total allowable catch 
was increased in both fisheries in 1998. 


4.53 We can also point to examples 
where progress is being made in the stock 
assessment process. In the offshore scallop 
fishery, science and industry are working 
together on an ongoing basis to understand 
the status of scallop stocks and to set 
harvest levels that protect juvenile 
scallops coming into the fishery. The 
Department’s Science Branch and industry 
monitor the catch of offshore scallops 
during the fishing season, which allows 
for harvest levels to be adjusted when and 
where necessary. In certain Gulf snow 
crab fishing zones, data from scientific 
surveys, landings and at-sea observers are 
used to provide industry with detailed 
information on location of the resource, its 
size and composition. 


4.54 Why is there an imbalance in the 
quality and availability of knowledge 
about individual stocks? Certain stocks, 
such as the Northern shrimp and 
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Newfoundland snow crab, are now found 
in areas and amounts that far exceed past 
experience; therefore, the amount of 
historical scientific information available 
is limited. The increase in these stocks 
likely reflects an absence of natural 
predators, primarily cod, and/or 
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favourable environmental conditions. For 
other stocks, such as lobster, some of the 
Department’s previous funding or 
management decisions have limited its 
knowledge-gathering capabilities. 
However, we note that in certain fisheries 
where co-operation between the 


Inshore Bay of Fundy Scallop Fishery —- Has “Conservation” Come First? 


In October 1986, the Minister established 
a line permanently separating the inshore and 
offshore Bay of Fundy scallop fishing fleets. 
Inshore vessels were gradually phased out of 
the offshore fishing grounds. The inshore 
fishing grounds experienced a rapid increase in 
landings, reaching record highs in 1989, but 
subsequently dropped drastically. 


Conservation problems identified. In a 
1995 meeting between the Department and 
industry representatives, departmental 
scientists indicated that management measures 
in place from 1989 to 1995 had been insufficient 
to prevent recruitment and growth overfishing. 
Growth overfishing is indicated when reducing 
the number of scallops harvested will result in 
an increase in the yield of the scallop stock. 
Recruitment overfishing occurs when there is a 
depletion of spawning-age scallops to the point 
where they are insufficient to maintain the 
scallop stock. 


Scientists recommended that steps be 
taken to reduce recruitment overfishing by 
substantially decreasing the meat counts 
(number of scallops per 500 grams) and thus 
increasing the size and the age of scallops 
harvested. In addition, they recommended 
closing large portions of the Bay of Fundy 
scallop beds to increase the supply of 
reproducing scallops. It was also 
recommended that growth overfishing be 
reduced by closing the remaining areas of the 
Bay of Fundy on a rotational basis when a large 
number of small scallops were harvested. 


The Department recommended a 
management framework to industry. In 
August 1995, the Minister announced that 
industry had agreed to a decrease in the meat 
counts and the closure of key scallop beds for a 
one-year period. Despite these actions, 
recruitment and growth overfishing continued in 
the inshore Bay of Fundy scallop beds. 


In early 1996, it became apparent to the 
Department and to industry that measures they 
had taken were not sufficient to protect small 
scallops; and catches continued to fall. The 
Department began discussions with industry to 
develop a harvesting plan with conservation 
measures to reduce or eliminate recruitment 
and growth overfishing and to develop a fleet 
self-rationalization plan. During discussions 
within the Department, scientists indicated that 
the Bay of Fundy inshore stocks were in 
desperate shape and suggested that a 
consideration of only biological factors would 
indicate that the fishery should be closed. 


In August 1996, the Department decided 
that the fishery should be kept open in spite of 
the concern raised by scientists. The 
Department indicated as part of its decision that 
before the 1997 fishery would be opened, 
industry would have to develop a fishing plan 
that included specific conservation measures 
aimed at mitigating the effects of continued 
fishing. Industry would also have to develop a 
fleet self-rationalization plan. 


Changes to conservation measures 
proved ineffective but fishing continued. 
During 1997 and 1998, the Department's 
fisheries managers and the industry developed 
a plan that contained conservation measures 
intended to mitigate the effects of recruitment 
and growth overfishing. The scallop fishery was 
conducted in 1997 and 1998 with a number of 
conservation measures (eight fishing zones with 
individual total allowable catches, reduction in 
meat counts, closed seasons, etc.) included in 
the fisheries plans. However, a key 
conservation measure that would allow the 
Department to monitor fishing activity by zone 
(the use of so-called “black box” technology) 
was not implemented. Also, during the fishing 
seasons there was limited verification of meat 
counts, limited at-sea patrols, and weaknesses 
in the monitoring of landings at dockside. 
Departmental scientists indicated that catch 


declines and recruitment and growth overfishing 
continued in1997. The impacts on the fishery in 
1998 of not having adopted the key 
conservation measures are not yet clear. 


Industry has adopted a management 
regime based on individual transferable quotas 
that resulted in a reduction in the size of the full 
Bay fleet (the fleet that accounts for 
approximately 85 percent of annual landings) 
from 99 active vessels to 66. The effects of this 
rationalization on the inshore scallop stocks 
have not yet been determined. 


The Department allowed harvesting in 
a nursery area. As part of the negotiations to 
develop harvesting and management plans, in 
1996 and again in 1997 fisheries management 
allowed the inshore scallop fishers access to an 
area known as “No Man’s Land” in scallop 
fishing area (SFA) 29. Departmental scientists 
and some industry representatives 
recommended that the area be considered a 
broodstock area and be permanently closed to 
fishing. Departmental scientists believed that 
scallops produced by this broodstock would 
replenish the inshore scallop beds. In 
July. 1996, the Science Branch made the 
following recommendation: “Given the 
possibility for recruitment overfishing in the 
fishing beds, it is preferred that SFA 29 (west of 
Baccaro Bank) remain closed as a broodstock 
area for other beds. This is a risk aversion 
strategy consistent with the lack of information 
on these resources.” 


Our concerns. There is a history of 
growth and recruitment overfishing in this 
fishery and the Department has shown an 
inability to effectively monitor the activities of the 
fishing fleet. Therefore, the decision to allow 
fishing in “No Man’s Land” appears to be 
inconsistent with the Department's commitment 
to a precautionary approach to fisheries 
management. 
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Department and industry is strong and the 


industry pays for additional information, 


there is good knowledge about the status 
of the stock. This has been the case in the 
Gulf snow crab and offshore scallop 
fisheries. 


4.55 
introduced the Integrated Fisheries 
Management Policy. Integrated Fisheries 


Management of the fisheries 


In 1996, the Department 


Management Plans (IFMP) were to be 


Atlantic Lobster Fishery - Problems with Knowledge Gathering 


Lack of knowledge is not a new issue. 
In the mid-1970s, lobster fishers, fisheries 
managers and scientists were all concerned 
about a possible collapse of some lobster 
fisheries in Atlantic Canada: the stocks were 
heavily exploited and recruitment failure was 
seen as a definite possibility because egg 
production was considered low. Between 1975 
and 1995, the Department conducted five 
studies that examined, to varying degrees, the 
level of knowledge of lobster population 
dynamics. All the reports identified the need for 
the Department to conduct more research to 
better understand the role of biology, the fishery 
and the environment in determining 
reproductive success. It was only following a 
1995 Fisheries Resource Conservation Council 
report that major initiatives were undertaken to 
examine these questions. 


From 1975 to 1995, the Department 
implemented only limited changes to harvesting 
practices in some areas, despite concerns 
about high exploitation rates and low egg 
production. During this period, lobster landings 
increased to the highest level observed since 
1893, when statistics on this fishery were first 
collected. The resulting conundrum for the 
Department was that catches were at a 
century-high level while the available scientific 
data were predicting a possible recruitment 
failure. Departmental scientists concluded that 
increased landings were the result of increased 
fishing effort and increased productivity of the 
stock caused by other factors. The relative 
contribution of these other factors to increased 
stock productivity was unknown. 


Fisheries Resource Conservation 
Council report recommended areas for 
improvement. In 1994, the Minister of 
Fisheries and Oceans asked the Fisheries 
Resource Conservation Council (FRCC) to 
examine the question of conservation in the 
Atlantic lobster fishery and to recommend ways 
to improve the conservation of the stock. The 


FRCC examined the state of scientific 
knowledge of the lobster fishery and reported 
that although there was a significant amount of 
knowledge on lobster biology and ecology, 
many key elements were still poorly 
understood. 


The Council pointed out that the 
Department was unable to provide definitive 
answers to a number of key questions, 
including: 


e the level of egg production required to 
conserve the lobster stocks. Scientists could 
not estimate the number of lobsters in the stock 
or the number spawning. The FRCC 
suggested that the Department use eggs per 
recruit as an index of egg production, with an 
interim target of 5 percent of the eggs per 
recruit in an unfished population. It also 
recommended that scientists continue to 
develop other indices of egg production. 
Subsequently, the Department adopted as an 
interim measure the goal of doubling existing 
eggs per recruit for each lobster fishing area. In 
most fishing areas this results in a target that is 
less than that recommended by the FRCC. It 
has recognized that there may be a need for a 
more appropriate definition of egg production 
(for example, egg production per unit area) and 
practical indicators that are easy to monitor; and 


e _ the key stock recruitment relationships 
required to predict longer-term changes in stock 
size. Scientists have not fully identified the key 
ecological factors that contribute to lobsters’ 
reproductive success and the successful 
establishment of juvenile lobsters. However, 
scientists had conducted a 20-year study in a 
fishing community in Newfoundland that was 
not definitive but that with 20 more years of 
study may provide a stock recruitment 
relationship for this one area. 


Science limited by reduced funding 
and poor data. In 1995, the Science Branch 
set aside $1.1 million over three years for 


special research to address the priority areas 
identified by the FRCC. A departmental 
assessment indicated that this special research 
increased its understanding of the lobster 
population dynamics. However, a critical project 
on growth and reproduction, considered 
important to improving estimates of egg per 
recruit and egg per recruit based on a reference 
point, was greatly curtailed by lack of funds and 
reassignment of personnel. 


The Department conducts research on an 
annual basis to determine the status of the 
lobster stocks. Information on commercial 
fishing activity is a major element to support this 
research. In 1998 the Science Branch reported 
that for lobster fishing area 34, “the lack of 
landings data by area fished is a dangerous 
situation as it makes a full assessment of the 
(lobster) fishery impossible.” While the 
Department has taken steps to improve the 
quality of the statistics collected, there are 
concerns about the timeliness of the information 
(as noted in paragraph 4.78 of the chapter) . 


As alternatives to complete statistics on 
the commercial fishery, the Science Branch has 
used “index” fishers and at-sea sampling for 
information on the activities of the commercial 
fishery. Departmental budget cuts have 
reduced the number of samples that the 
Science Branch can obtain from these 
alternative sources. The reduced samples have 
increased the variability of the data collected 
and increased the uncertainty in the scientists’ 
assessment of the stock status. 


Our concern. Despite recent efforts by 
the Department, there continue to be gaps in its 
knowledge of lobster stocks. This limits the 
Department's ability to ensure that its 
conservation measures are working in this 
fishery. 
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developed for each fishery. These plans 
were intended to provide for meaningful 
participation by stakeholders 1n the 
management of the fisheries. They would 
also serve to co-ordinate the activities of 
the Department’s various branches in 
managing the fisheries. However, the 
IFMPs are not binding on the Minister, nor 
do they limit his absolute discretion in 
making resource allocation and licensing 
decisions. 


4.56 A review conducted for the 
Department was critical of the way the 
IFMPs have been completed. One specific 
problem the review noted was the impact 
of stakeholders going outside the IFMP 
process directly to the Minister. It also 
noted that key components of the 
Department (for example, Science, 
Conservation and Protection, and 
Aboriginal Affairs) were not fully 
integrated into the process; nor did 
external stakeholders feel that they were 
considered in the process. Further, the 
review noted that IFMPs were not 
prepared on a timely basis. Our audit 
findings support these observations. The 
Department has begun to implement a 
management plan in response to the 
review. We believe that integrated 
planning is important for cost-effective 
management of the fisheries. 


4.57 In our October 1997 Report 
Chapter 15, we observed that rules are 
needed for key decisions in order to help 
guide resource use. We noted that without 
biological reference points or decision 
rules as guideposts, social and economic 
factors could influence decisions on the 
use of individual stocks to the detriment 
of the biological component of 
sustainability. Such rules have generally 
not been developed for the use of the 
shellfish stocks we examined except, as 
noted in the case study on page 4-20, in 
the lobster fishery. If “conservation” of the 
stocks is the primary objective, then 
fisheries managers need to work with 
industry and scientists to develop rules for 
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key decisions on resource use and include 
these rules in fisheries management plans. 


4.58 The offshore scallop fishery uses 
a “roll-over total allowable catch” concept 
— that is, harvesting activity is adjusted 
on an ongoing basis according to catch 
rates and composition of the stock. This 
concept, which is included in the formal 
fisheries plan, is designed to limit pressure 
on juvenile scallops and, therefore, on 
their future recruitment into the fishery. 
This is a fishery where industry and 
science have developed precautionary 
rules for decisions on use of the resource. 


4.59 We found other examples where 
conservation goals exist but are not part of 
the formal fisheries plan. These goals are 


It is not clear how 
fisheries managers 
plan to incorporate the 


in the form of reference points but have precautionary 

not been formally adopted as decision : we 
rules. In the Gulf snow crab fishery, for approach into decision 
example, a predetermined exploitation making. 


rate is used in conjunction with biomass 
estimates to determine the total allowable 
catch. However, this exploitation rate does 
not have a strong biological basis, as it 
was originally chosen for economic 
reasons. The Gulf shrimp fishery uses a 
“basket” of biological indicators to 
recommend the direction of change in the 
yearly total allowable catch. 


4.60 The Science Branch is currently 
developing a working definition of the 
precautionary approach and incorporating 
it into its advice to the Fisheries 
Management Branch. We support 
movement in this direction. Still, it is not 
clear how fisheries managers plan to 
incorporate the precautionary approach 
into their decision making on resource 
use. 


4.61 The Fisheries Management 
Branch is not compelled to follow the 
advice that emerges from the stock 
assessment process. In some fisheries 
there is a direct correlation between the 
stock assessment advice and the fisheries 
management decisions. In other fisheries, 
considerations other than stock assessment 
advice play a significant role in decisions 
on resource use. In fact, we found that in 
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some such decisions it was difficult to see 
how fisheries managers had applied a 
precautionary approach: 


¢ In 1997 changes were made to key 
conservation measures planned for the 
Bay of Fundy scallop stocks. These 
measures — aimed at protecting young 
scallops — were changed after industry 
and political concerns had been expressed 
about their impact on industry viability. 


e The Newfoundland snow crab 
management plan, approved in 1997, was 
intended to be in place for 1997 and 1998 
without a change in the total allowable 
catch, unless the scientific advice 
indicated that there was a substantial 
change in stock status. On 20 March 1998, 
the Regional Science Branch advised the 
Regional Fisheries Management Branch 
that there was no basis for suggesting a 
change to the management plan for 1998. 
However, on 3 April 1998, Newfoundland 
fisheries managers notified the scientists 
that they had made a commitment to 
review the 1998 total allowable catches 
for the inshore fleet. At that time, fisheries 
management proposed increases totalling 
6.7 percent to address this commitment to 
the inshore fleet. The Regional Science 
Branch then concurred with the proposal 
on 6 April 1998. The total allowable catch 
was subsequently increased by 
9.5 percent. 


e The Science Branch did not consider 
the doubling of the 1998 total allowable 
catch in shrimp fishing area 6, which 
followed the doubling of the total 
allowable catch the year before, to be a 
“conservative or precautionary approach”’. 
Further, as noted in paragraph 4.52, there 
was no formal Stock Status Report to 
support the 1998 decision. 


4.62 The purpose of the Integrated 
Fisheries Management Plans 1s to 
integrate the interests of all stakeholders 
in the management of the fishery. 
However, we have observed examples of 
planning that did not deal with scientists’ 
concerns about unsustainable practices. 


4.63 For example, before the inshore 
sector began fishing for Northern shrimp 
in 1998, the Science Branch and others 
had raised concerns about the likelihood 
that large quantities of small, low-priced 
shrimp would be dumped at sea by this 
sector. Such practices waste the resource 
and affect the future growth of the stock. 
It would be important that the Science 
Branch have access to information about 
the extent of such practices when it makes 
assessments of the stock status. However, 
the 1998 plan for this fishery did not 
reflect scientists’ concerns about dumping 
at sea. 


Monitoring, control and surveillance — 
ensuring the sustainable harvest of the 
stocks 


4.64 Monitoring, control and 
surveillance (MCS) refers to the systems 
and processes used to capture information 
about the commercial fishery and, in some 
instances, to control the fishing practices 
of industry. In the Atlantic shellfish 
fisheries the Department conducts and 
funds some MCS, while other aspects are 
delivered by private sector organizations 
and paid for by industry. Exhibit 4.5 
shows the MCS systems and processes 
used in the Atlantic shellfish fisheries. 


4.65 The objectives of MCS include 
gathering complete and accurate 
information about the fishery and ensuring 
that the fishery is conducted in accordance 
with the fisheries management plan and 
the fisheries regulations. Co-ordinated and 
effective MCS, including effective 
enforcement, is important to ensure that 
fishers are conducting their harvesting 
activities in a sustainable manner. 


4.66 We observed that in most 
fisheries there is a lack of co-ordination in 
planning MCS. Responsibility for aspects 
of MCS is split among the Department’s 
various components. Despite the use of 
Integrated Fisheries Management Plans in 
many fisheries, the various components of 
the Department operate largely in 
isolation of each other. The more complex 
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the fishery, the less likely it is to have a 
co-ordinated approach to MCS. For 
example, the Northern shrimp fishery is 
one that includes both offshore and 
inshore fleets, covers a large geographic 
area and involves more than one region of 
the Department. Yet we found that the 
Plan for this fishery did not address 
concerns about dumping at sea by the 
inshore sector. 


4.67 In comparison, the Gulf crab 
fishing area 19 fishery covers a small area 
that has few landing locations, a small 
number of participants and an industry 
group that is active in decisions on the 
management of the fishery. These factors, 
combined with the limited number of 
fishery officials involved, make it easier 
to implement effective MCS, including 
enforcement. 


4.68 Our October 1997 Report 
Chapter 15 discussed elements of MCS in 
the groundfish fishery. The following 
observations include our comments on the 
Department’s progress in areas reported in 
that chapter, since the practices in the 
shellfish and the groundfish fisheries are 
somewhat similar. 


4.69 Dockside monitoring. Dockside 
monitoring is the process used in most 
fisheries, except the lobster fishery, to 
gather information about landings by 
individual fishers. Fishers procure the 
services of a dockside monitoring 
organization to monitor their landings of 
fish and report them to the Department. 


Fisheries and Oceans 


At-sea enforcement 
Aerial surveillance 
Dockside enforcement 


Other surveillance 


Data analysis 


Shellfish Fisheries 
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4.70 In our October 1997 Report, we 
observed that while this system was an 
improvement over the previous system, 
there were weaknesses in controls over 
dockside monitoring. The Department 
responded in part, by enacting regulatory 
changes designed to control the activities 
of dockside monitoring organizations; 
these came into force | January 1999. We 
observed that the problems in dockside 
monitoring for groundfish also applied to 
shellfish. 


4.71 In 1998, enforcement officers 
were requested to conduct “audit” tests of 
dockside monitoring activity to provide 


information about its overall effectiveness. 


Only the Newfoundland Region attempted 
to conduct these audits. It encountered 
problems and was not able to obtain the 
desired information about the operation of 
dockside monitoring in Newfoundland. 
Still, we are encouraged that the 
Newfoundland Region made this initial . 
attempt and that in the 1999 fishing year it 
appears to be changing its approach to the 
“audit” of dockside monitoring. 


4.72 At-sea observers. There are 
several methods available to the 
Department to monitor and control fishing 
practices at sea, although at-sea observers 
on fishing vessels or boarding by 
enforcement officers are the methods that 
provide first-hand insight. The main 
practices that these methods address are 
dumping and discarding at sea or using 
illegal harvesting methods. 


Private Sector 
( gathers information and 
reports to Department) 


At-sea observers 


Dockside monitoring — 
gathers information about 
landings by individual fishers 


Conservation Harvesting Plans 


Exhibit 4.5 


Monitoring, Control and 
Surveillance Systems Used in 
the Atlantic Shellfish Fisheries 
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Information from 


at-sea observers was - 


not used effectively to 


4.73 In October 1997, we noted that 
the information provided by observers was 
not being used effectively to determine the 
extent of unsustainable fishing by the 
groundfish industry. In addition, the 
Department had not developed a 
systematic approach that integrated 
information from at-sea observers and 
other sources into its management 
practices. These observations also applied 
to shellfish management in 1998. 


4.74 Coverage targets — 

the percentages of fishing trips planned 
with at-sea observers on board — vary 
among fleets, from no coverage in the 
lobster fishery to 100 percent coverage in 
the offshore Northern shrimp fishery. 
Moreover, different targets exist even for 
fleets fishing the same stock: 


e Coverage targets for Northern shrimp 
are 100 percent of the offshore sector and 
10 percent of the inshore fishers. 


e For Gulf snow crab, coverage targets 
are 20 percent and 5 percent in areas 12 
and 19 respectively, although these are 


e Newfoundland snow crab has no 
observer coverage of the inshore 
“temporary” fleet, but a target of 5 percent 
coverage of the rest of the fleet. 


We could find no systematic basis for 
establishing coverage targets for at-sea 


4.75 In some fisheries, such as Gulf 
snow crab and Newfoundland snow crab, 
targets for fleet coverage by observers 
were met in 1998. Even so, only 

1.5 percent of the actual catch of 
Newfoundland snow crab was observed. 
In the Northern shrimp fishery, the inshore 
fleet in shrimp fishing area 6 attained 

5 percent coverage in 1997 and 6 percent 
in 1998; the target for each year was 


While some information from 
at-sea observers is used to support science 
or enforcement activity, we found that for 


the most part it was not used effectively to 
manage the fishery. For example, 
management ignored some observer 
information that had been collected 
specifically to address key conservation 
concerns. In 1998, the Gulf snow crab 
area 18 fishery had a protocol established 
for conditions that required a closure. If 
the amount of landed crab that was 
soft-shelled (recently molted) exceeded 
20 percent of the catch over a 48-hour 
period, industry was advised to modify its 
practices. If the same conditions existed in 
the subsequent 48-hour period, the fishery 
was to be closed for the season. We noted 
several occasions when the incidence of 
soft-shelled crab exceeded 20 percent for 
two consecutive 48-hour periods without 
the closure of the fishery. We noted that 
the fishery was closed three times during 
the season and subsequently reopened. We 
also observed that as the incidence of 
soft-shelled crab increased, the amount of 
at-sea coverage decreased. In fact, 
observer coverage was lowest when 

the percentage of soft-shell crab was 
highest. In the Newfoundland snow crab 
fishery, at-sea observer information on the 
incidence of soft-shelled crab is not used 
to determine whether closure is warranted. 


4.77 In October 1997, we noted that 
the Department was not using information 
from at-sea observers to determine the 
extent to which the groundfish industry’s 
practices were unsustainable. This type of 
analysis is referred to as “indexing” — the 
practices and catches of vessels with 
at-sea observers are compared with those 
of the other vessels. This gives an 
indication of whether fishers engage in 
unsustainable practices when observers 
are not on board. We found in the shellfish 
fishery, too, that as a rule the “indexing” 
analysis is not completed. The two 
exceptions were the Newfoundland snow 
crab fishery (1996 and 1997) and Gulf 
snow crab area 12 (1998). The “indexing” 
analysis showed the occurrence of 
highgrading (retention of only 
higher-value snow crab) and discarding — 
25 percent in the Newfoundland snow 


manage the fishery. 
adjacent areas. 
observers. 
10 percent. 
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crab fishery and 20 percent in Gulf snow 
crab area 12 fishery. 


4.78 Fishery management 
information systems. The observations 
we made in 1997 about the fisheries 
management information systems are still 
valid. These systems are of limited use to 
those engaged in fisheries management. 
There is a lack of integration between the 
various systems, and the information is 
not timely or complete. For example, 
information on lobster landings for 1997 
was still not available by December 1998. 
The Department has a five-year plan to 
replace these systems. 


4.79 Aerial surveillance. Aerial 
surveillance is heavily concentrated on 
fisheries near the international boundaries, 
with limited coverage of the shellfish 
sector. It is used to provide management 
with information on industry activity, to 
help target enforcement activities, and to 
monitor and enforce specific zone areas. 


4.80 Other surveillance. The 
Department is experimenting with other 
types of innovative surveillance 
techniques. For example, the offshore 
scallop fleet has adopted so-called “black 
box” technology, which uses global 
positioning systems to monitor activity. 
This system provides the Department with 
a real-time monitoring capability. 


4.81 Enforcement. We observed that 
enforcement activity is more rigorous in 
some fisheries than in others. The reason 
for this is not always obvious but depends 
on the circumstances in each fishery. For 
example, there is a relatively high level of 
enforcement in the Gulf snow crab area 19 
fishery. 


4.82 By contrast, there is minimum 
enforcement coverage in the Bay of Fundy 
scallop fishery. Enforcement officials 
believe that putting more resources into 
monitoring this fishery would not be 
productive, given the combination of the 
existing fisheries management plan and 
the lack of at-sea enforcement capability. 
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In fact, the post-season review of 
conservation and protection for this 
fishery states that the existing fisheries 
management regime cannot be enforced. 


4.83 We found that there are gaps in 
enforcement coverage at sea because the 
Department does not have the vessels to 
cover certain areas. For example, 
midshore and offshore shellfish areas have 
little or no coverage by patrol vessels. 
Near-shore coverage (closer than 

20 miles) has recently improved. Since 
1996, the Department has purchased 

69 specialized vessels to cover this area. 
In areas where there is limited at-sea 
enforcement, at-sea observer coverage 
becomes the principal tool to control 
unsustainable harvesting practices such as 
discarding and dumping; however, as we 
have already noted, for the most part this 
tool is not being used effectively. 

Exhibit 4.6 shows, for shellfish stocks 
included in our audit, the areas where the 
Department has limited or no capability to 
perform at-sea enforcement. 


4.84 In 1998, the Department 
conducted forensic audits and 
enforcement actions in several fisheries, 
including snow crab fisheries in Nova 
Scotia. These enforcement actions have 
provided an insight into certain 
unsustainable practices by fishers and 
those who buy their fish, and have led to 
the laying of a number of charges. Up 
until now, the Department has relied 
heavily on the skills of a few individuals 
to conduct its forensic audits. However, it 
is working to develop the institutional 
capability to continue this important 
activity. 


4.85 Administrative sanctions and 
penalties. Under the Department’s 
Administrative Licence Sanction Policy, 
the Regional Director General could apply 
a sanction (for example, suspend the right 
to fish for a period of time) in addition to 
criminal prosecution in the event of a 
serious conservation-related offence. As 
we noted in October 1997, the Department 
has put the sanction policy on hold 
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Exhibit 4.6 


Shellfish Stocks Audited - 
Areas with Little or No At-Sea 
Enforcement Capabilities 


Lobster 
Scallop 
Shrimp 
Snow Crab 


Source: Review of Fisheries 
and Oceans data 


pending the hearing by the Federal Court 
of Appeal of several court decisions made 
by the Federal Court—Trial Division, 
which held that the Minister did not have 
the authority to impose administrative 
sanctions. Some form of sanction can be a 
cost-effective tool because it can be 
tailored to the nature of violations and can 
be applied in a timely manner. 


Summary 


4.86 The Department has not set out 
what it means by “conservation” in a way 
that would guide its managers when 
making resource use decisions. While 
fisheries managers understand that the 
“conservation” objective reflects the 
biological aspects of sustainability, most 
fisheries management plans we reviewed 
did not contain objectives that were clear 
and measurable. We found weaknesses in 
the information available to support 
resource use decisions. The Department 
does not have biological reference points 


or guidelines for key decisions on the 
biological component of sustainability. 
Gaps and weaknesses in the monitoring, 
control and surveillance systems and 
processes limit their operational 
effectiveness. 


4.87 The Department should 
develop, over time and in consultation 
with fishers and other stakeholders, 
biological reference points and 
conservation guidelines as the basis for 
making recommendations on resource 
use decisions. 


Department’s response: Fisheries and 
Oceans has established a Working Group 
that has been given the responsibility of 
clarifying its Atlantic fisheries 
management policies. In the context of 
this initiative, a number of broad 
management aspects will be addressed, 
including better defining conservation and 
the precautionary approach. Once these 
definitions are adopted, this would lead to 
the establishment of biological reference 


Labrador 


REGIONS 

A — Scotia Fundy 
B - Gulf 

C — Laurentian 

D — Newfoundland 
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points and of rules for taking 
conservation-related decisions. 


4.88 The Department should correct 
weaknesses in its monitoring, control 
and surveillance systems and processes. 


Department’s response: Under the 
framework developed by DFO for the 
preparation of Integrated Fisheries 
Management Plans (IFMPs), it is provided 
that they are to include enforcement 
measures for the control and monitoring of 
fishing activities to achieve management 
objectives. Many of the existing IFMPs for 
shellfish fisheries already include such 
measures and any new plan being 
developed along the IFMP framework will 
provide for such measures. Any gap or 
weaknesses in the surveillance systems are 
addressed as they are brought to the 
attention of the responsible managers. 


Co-management — An Important 
Initiative 


4.89 The Fishery of the Future 
strategy has called for the “devolution to 
fish harvesters of a greater 
decision-making role and more 
responsibility for costs of resource 
conservation and management”. Initially, 
the government proposed amendments to 
the Fisheries Act that promoted a form of 
partnering as the way it would structure 
the devolution of decision making. These 
amendments did not receive final 
parliamentary approval and died on the 
Order Paper. The Department eventually 
decided to move forward with a different 
concept, called co-management. 


What is co-management? 


4.90 At the most basic level, 
co-management is a form of power 
sharing. The extent and form of this power 
sharing can be quite varied and can evolve 
over time. Co-management involves an 
agreement between at least one level of 
government and another group. A 
definition of co-management has been put 


Report of the Auditor General of Canada — April 1999 


Fisheries and Oceans — 
Managing Atlantic Shellfish in a Sustainable Manner 


forward by the National Round Table on 
the Environment and the Economy 
(NRTEEB), in its paper Sustainable 
Strategies for Oceans: A Co-Management 
Guide: 


Co-management is a system that 
enables a sharing of decision-making 
power, responsibility, and risk 
between governments and 
stakeholders, including but not 
limited to resource users, 
environmental interests, experts, and 
wealth generators. 


4.91 The NRTEE has concluded that a 
co-management approach to ocean issues, 
including fisheries management, has 
potential benefits that include bringing 
together interests, changing relationships, 
fostering joint accountability, supporting 
transparency and autonomy, devolving 
decision making and responding to 


regional needs. Co-management is a 


The Department’s approach to form of power sharing. 


co-management 


4.92 The Department’s “Framework 
and Guidelines for Implementing the 
Co-management Approach” (17 April 
1997) describes four possible levels of 
co-management. The first and most basic 
level that the Department considers to be 
co-management has user groups providing 
input to the Integrated Fisheries 
Management Plans. The second level has 
user groups, through their legally 
constituted, representative organizations, 
entering into agreements that reflect a 
greater involvement in the management of 
their specific fishery. At the third level, 
fishers would enter into formal partnering 
through legally binding arrangements that 
transfer greater responsibility to industry. 
This level would require changes to the 
existing Fisheries Act. The fourth level is 
co-management legislated under land 
claims settlements. 


4.93 In the fisheries that we examined, 
only the first and second of these levels of 
co-management were represented. The 
third level has not been pursued because 
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The Department has 
not defined its core 
activities. 


the amendments to the Fisheries Act did 
not receive final parliamentary approval. 
The Northern shrimp fishery conducted 
within the bounds of the Nunavut 
Settlement Area is subject to the Nunavut 
Land Claims Agreement; this is a form of 
legislated co-management. Our audit did 
not examine this particular arrangement. 


4.94 The co-management 
arrangements that we examined involved 
no sharing of real decision making but 
they did formalize the process of 
consultation with stakeholders. Also, there 
was no broad-based stakeholder 
involvement; instead, the arrangements 
focus on already identified licensed fishers 
or their organizations. 


Is it co-management or offloading of 
costs? 


4.95 In May 1996, the Department 
issued a paper entitled Fisheries 
Management Partnering Policy Principles. 
Among other things, this paper sets out 
the principle of cost recovery: 


... all resource management costs that 
are attributable to the fleet and that 
result in or support private benefit to 
the fleet should be either paid for or 
undertaken by the fleet. Those costs 
which can be associated with public 
benefits would be paid for through 
appropriations. This will require the 
calculation of attributable costs for 
fisheries supported by a credible 
departmental costing system. 


4.96 We found that the Department 
has not determined which of its resource 
management activities, including science 
activities, result in or support private 
benefit to the various fleets. In other 
words, it has not consistently made the 
distinction between its core activities and 
other activities. In addition, it does not 
have a costing system that could generate 
this kind of information. 


4.97 In reality, the types of costs 
recovered from industry are quite varied. 
In some sectors, such as offshore scallops, 


the industry itself pays directly for the 
cost of certain fisheries science activities. 
In other sectors, costs are recovered from 
the industry but there is no consistency in 
the types of costs that are recovered. For 
example, in Gulf crab fishing areas 12 and 
19 a majority of the costs of stock 
assessment have been recovered from 
industry. By contrast, in the Gulf shrimp 
fishery only science costs that were 
incremental to existing arrangements were 
recovered. 


4.98 It is useful to look at the fisheries 
where industry does not pay any of the 
costs of fisheries management and 
science. There, for example, the 
Department funds the costs of the annual 
stock assessment process, including any 
research surveys. Yet in 1996, when the 
Department wanted to enter into a 
co-management arrangement with fishers 
in Gulf snow crab fishing area 12, it asked 
them to pay the costs of stock assessment, 
including the research survey. When the 
industry refused to pay these costs because 
of a dispute with the Department over the 
sharing of the 1996 quota, the Department 
did not carry out the survey for that year. 
In the following year, the industry funded 
the research survey. Given its 
“conservation” objective it is not clear 
why, in this case, the Department has 
considered the stock assessment process, 
including the research surveys, to be a 
private rather than a public benefit. 


4.99 Co-management, as it has been 
applied, has added another layer of 
management costs to the existing 
structure. A panel appointed by the 
Minister to study the partnering concept 
reported on 10 December 1998, “Those 
who have negotiated a co-management 
agreement with DFO admit that the 
process was both time-consuming and 
costly.” In addition, the panel reported 
that all of the people it consulted outside 
the Department felt that co-management 
represented a downloading of fisheries 
management costs from the Department to 
industry. The Department has not 
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determined whether the total cost of 
managing the fishery, for both industry 
and government, is higher or lower under 
co-management. 


4.100 In paragraph 4.24, we noted that 
fishers organizations were given 
temporary allocations of shellfish to 
strengthen or support initiatives by the 
organizations and to redistribute income to 
their members. Before approving the 
temporary allocations, the Department had 
the fishers’ organizations submit a 
business plan for fishing the allocation 
and using the resulting benefits. In certain 
fisheries (for example, Gulf crab fishing 
area 12), fisher organizations that received 
the temporary allocations charged fees or 
imposed other conditions for access to a 
portion of the allocation. Once the 
allocations were granted to the fisher 
organizations, the Department had no 


mechanism to hold them accountable for 
following the business plans they had 
submitted. 


Arrangements with third parties 


4.101 We have concerns about the way 
the Department implemented financial 
arrangements with fisher organizations. 
The case study below describes our 
concerns about the imposition of a fee for 
access, totalling $5 million in 1997 and 
$2 million in 1998, that may not be 
contemplated in the legislation. 


4.102 We also have concerns about the 
way the Department is using a specified 
purpose account to reimburse its own 
departmental expenditures. The Financial 
Administration Act requires all public 
moneys to be deposited to the credit of the 
Receiver General. These funds generally 
are recorded in the Accounts of Canada as 


Solidarity Funds - Imposition of a Fee That May Not Be Contemplated in Legislation 


In 1997, changes to allocations for Gulf 
crab fishing area 12 resulted in the 
displacement of many plant workers and crew 
members. In response, a fund was established 
to provide employment opportunities for those 
affected. According to departmental documents 
and press releases, this fund (Solidarity Fund) 
was to be voluntarily established by industry 
and was to be operated at arm’s length from 
the Department. Initially, a fund was to be 
established for New Brunswick only, but 
pressures from crab fishers in other provinces 
led to the creation of similar funds in Quebec, 
Nova Scotia and Prince Edward Island. 


Fishers in area 12 contributed $5 million 
in 1997 and $2 million in 1998 into the funds 
through a levy on the price paid per pound of 
snow crab landed. Fishers each paid to their 
fishers organization a fee of $0.15 per pound in 
1997 and $0.08 per pound in 1998. For 
example, a fisher who was allocated 300,000 
pounds was required to pay $45,000 in 1997 
and $24,000 in 1998. 


In both 1997 and 1998, the Department 
allocated quotas to companies established by 
the fishers’ organizations. In each year, 


approximately 20 percent of the snow crab that 
had been originally allocated by the Minister to 
the traditional midshore fishers was reallocated 
by the Department to these organizations. 
Departmental officials have informed us that a 
majority of the traditional midshore fishers 
approved, by a vote, the overall 
co-management approach, including 
establishment of the solidarity funds. 


Subsequently, the Department approved 
the transfer of this 20 percent allocation from 
the fishers’ organizations to the individual 
fishers. However, the Department approved 
transfer requests from the fishers’ organizations 
only after receiving confirmation that the 
individual fishers had paid the per pound levy to 
the organization and that the moneys were on 
deposit in an “in trust’ account created at a 
financial institution (in trust pending transfer to a 
solidarity fund). Release of moneys from the “in 
trust” account to the solidarity fund was not 
completed until the financial institution 
received “notification from the Department of 
Fisheries and Oceans (DFO) that a legal 
entity or instrument that meets the 
requirements/satisfaction of DFO has been 
created”. 


Prior to authorizing transfer of money to 
the solidarity funds, the Department required 
business plans describing how the money 
would be used. It also made suggestions on the 
funds’ structure and mandate. In New 
Brunswick, a departmental official sits on the 
fund’s Board of Directors and influences 
decisions on projects to be financed with fund 
money. 


Our concerns. The Department is 
involved in fund decision making and in 
ensuring that the levy is paid and transferred to 
the solidarity funds. In our opinion, through 
these actions it has effectively imposed a fee on 
fishers to access the resource, which does not 
appear to be contemplated in legislation. 


In addition, the Department has no 
mechanism to report on its activities with 
respect to these funds. The Department has 
stated that it is operating at arm’s length from 
the funds and, therefore, in our view is not 
acting in a transparent and accountable fashion. 
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revenue. However, there are exceptions to 
this requirement. For example: 


e Parliament may give a department 
the authority to apply such funds against 
its appropriation as a negative expenditure 
(‘‘net-voting authority”). 


e The Receiver General for Canada 
may give a department the authority to 
establish a specified purpose account (an 
account in the Consolidated Revenue 
Fund) to receive and disburse moneys for 
specific activities. 


4.103 Treasury Board policy requires 
that specified purpose accounts must not 
be used to reimburse departmental 
expenditures. However, goods and 
services provided by departments may be 
purchased with specified purpose funds 
when: 


e the department providing the goods 
or services is not otherwise involved in the 
use of the money (that is, it is not part of 
the cost sharing or joint project 
agreement); and 


e the department providing the goods 
or services operates under a revolving 
fund or net-voting authority. 


4.104 Fisheries and Oceans 1s 
disbursing money from a specified 
purpose account to reimburse its own 
departmental expenditures (that 1s, the 
salaries of scientists, conservation and 
protection officers, and fisheries resource 
managers) incurred under a joint project 
agreement with third parties. The 
Department does not have a revolving 
fund and does not have net voting 
authority. In our view, the Department’s 
management of the specified purpose 
account is not consistent with the Treasury 
Board policy that governs this activity. 


Summary 


4.105 It is important that the 
Department move away from 
micro-management of the fishery to a 
form of industry participation or decision 


making that clearly articulates 
accountability arrangements. 
Co-management arrangements that we 
examined are largely cost-sharing 
arrangements and have involved no 
sharing of real decision making. 


4.106 The Department should clarify 
its objectives for co-management and, 
where necessary, seek parliamentary 
approval to implement this approach. 


Department’s response: Fisheries 
Management co-management is about 
DFO and fishers working together to 
better manage Canada’s commercial 
fishery. This concept is at an early stage 
and many pilot projects have been 
undertaken. The co-management approach 
builds on a renewed relationship between 
government and the fishing industry. The 
approach we have developed is the 
co-development, with industry, of an 
Integrated Fisheries Management Plan, 
which sets out harvest levels, conservation 
requirements and certain allocation 
processes for participants. This is 
undertaken under the authority of the 
Fisheries Act. As well, and on a voluntary 
basis, Joint Project Agreements are 
negotiated with fishers or their 
representatives. These are legally binding 
agreements and are not directly related to 
the IFMP. The legal authority is the 
Financial Administration Act. Control 
measures are in place within the 
Department to ensure financial and legal 
aspects are respected before concluding 
these arrangements. 


Although the first recommendation of the 
Panel on Partnering urges the Minister of 
Fisheries and Oceans not to go forward at 
this stage with legislation for partnering, 
its second recommendation is to urge the 
Minister and DFO to pursue 
co-management and partnering as an 
important building block for the fishery of 
the future. DFO will pursue the 
co-management approach with interested 
groups. 
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Conclusion 


4.107 We noted significant weaknesses 
in the Department’s management practices 
designed to achieve its objectives for the 
Atlantic shellfish fishery. Our audit found 
decisions that contradict the Department’s 
Fishery of the Future strategy, which 
formed the basis of our criteria for this 
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audit. In addition, the Department is 
pursuing social objectives that it has not 
articulated to Parliament, and economic 
objectives for which it has not identified 
expected results. There is an urgent need 
for the Department to clarify these 
objectives and to develop and implement 
the strategies to achieve overall 
sustainability of the Atlantic shellfish 
fisheries. 
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About the Audit 


Objective 


The objective of our audit was to assess the extent to which the Department’s fisheries management practices 
in the Atlantic shellfish fisheries support and complement its management objectives. 


Scope and Approach 


In October 1997 we reported on weaknesses in the management of the Atlantic groundfish stocks, and 
chronicled the problems that eventually led to their collapse. While groundfish landings are at historical lows, 
the shellfish industry is currently recording historically high landings. Lobster and scallop fisheries have a 
long history in Atlantic Canada but the importance of other shellfish fisheries is a more recent phenomenon, 
with some having become important only in the 1980s. Since then, the landed value for most shellfish has 
risen dramatically. 


The subject of this audit was the Department’s fisheries management practices in the Atlantic shellfish 
fisheries. Our approach was to assess whether the Department’s management of the shellfish industry has 
been designed and operated to meet its stated objectives for fisheries management, including related science 
functions. 


We examined the lobster, snow crab, shrimp and scallop fisheries to determine whether and to what extent the 
fisheries management objectives have been implemented in practice. In addition, we reviewed fisheries 
management practices to determine whether they are consistent with the management objectives. 


We conducted this audit through interviews with departmental staff and representatives of industry. We 
examined and analyzed departmental files, data and documentation. We observed enforcement officers 
carrying out their activities in ongoing fisheries. 


Criteria 
We expected that the Department’s fisheries management practice would: 


e be designed and implemented to achieve the biological aspects of a sustainable resource and to assure 
sustainable utilization of the resource; 


e include sharing responsibility with industry, where appropriate and within the limits of existing 
legislation for fisheries management and decision making, while holding it accountable for the 
agreed-upon sustainable use objectives; and 


e support the Department’s strategy to make fisheries economically viable and self-reliant over time. 
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Appendix 


Good Practices Review — Sustainable Fisheries Framework 


The history of world fisheries provides many examples of unsustainable fisheries. Over and over again, significant 
world fish stocks have been reduced to levels far below what could be considered sustainable. In fact, the Food and 
Agriculture Organization (FAO) of the United Nations has reported that 60 to 70 percent of the world’s fish stocks 
require urgent intervention to avoid further decline of fully exploited and overfished resources and to rebuild depleted 
stocks. 


As part of this audit, we reviewed fisheries frameworks in other jurisdictions and material written on sustainable 
fisheries by notable organizations such as the FAO, to determine if there were lessons to be learned for our Canadian 
context. It should be noted that the lessons learned represent the strengths we observed in all of the fisheries 
frameworks of the jurisdictions we reviewed, including Canada. They do not represent observations about the fisheries 
regime of only one or two jurisdictions. 


Our intention in preparing this review of good practices is not to provide a prescriptive approach to fisheries 
management. Rather, we believe that it is important to note those practices that appear to support sustainable fisheries. 


Lessons Learned 


Legal framework for sustainability. We observed jurisdictions where the legislation governing fisheries management 
reflects important concepts of sustainability. For instance, the means by which the biological, economic and social 
factors affecting the fishery are to be considered are established in a clear and transparent fashion. Other important 
factors set out in legal frameworks include conservation goals, rules over access, issues of property and use, 
enforcement, and dispute resolution. In addition, the legislative bodies of certain jurisdictions have put a good deal of 
ongoing effort into considering and debating the legal framework needed to support a sustainable fishery. 


Science and precaution. The objectives of those responsible for managing and of those fishing should be to 
continually expand the understanding of the individual stocks but also to understand the important interactions of 
species in the ecosystem. Included in this understanding is the ability to take into account natural environmental 
fluctuations in the ecosystem. We observed operational decision rules based on biological reference levels that have 
been developed from an understanding of the affected stocks and ecosystems. In recognition of the limits to scientific 
approaches, management regimes can incorporate methodology that uses a precautionary approach when confronted 
with uncertainty. It should be recognized that the precautionary approach is a developing concept and that much of the 
work on it has been undertaken by scientists rather than by fisheries managers, yet it is the fishery management that 
has to be precautionary, not the science. Harvest amounts respect the biological reference level where possible, and 
strong monitoring is undertaken to ensure that the harvest amounts are respected. Knowledge of changes in the 
ecosystem is important in considering their impacts on the harvest, although it is recognized that this is a developing 
area. 


In most jurisdictions, knowledge-gathering processes depend very heavily on biology while underusing the social and 
economic sciences. We did observe in at least one jurisdiction the use of economics in establishing important decision 
rules and ecosystem impacts. The involvement of industry and affected communities in decision making could lead to 
a higher level of involvement of these other sciences in knowledge gathering. 


Vested interest of participants. Successful fisheries management regimes ensure that all participants have a vested 
interest in promoting and achieving a sustainable fishery. A primary vested interest is the economic viability of 
industry. 
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Responsibility of industry. We noted that those who receive the right to fish are held responsible in some jurisdictions 
for any of their activities that are not sustainable. Ultimately, the penalty they pay for unsustainable behavior 
outweighs any benefit of such behavior. 


Industry plays an important role in decision making in the fishery, whether in a formal or informal way. 


A strong fisheries management institution. In some jurisdictions, the government institution responsible for 
protecting the public interest in the resource can hold accountable those given the right to fish. In several jurisdictions, 
the cost of fisheries management is either partially or fully recoverable from the industry that benefits from the 
resource. The government institution is held accountable for the cost-effective delivery of a specifically defined 
fisheries management function. 


In addition, the government institutions are supported by clearly articulated principles for resource allocation 
decisions. The decisions are made in a transparent and equitable fashion and are subject to challenge. 


Government subsidization. The FAO has promoted a policy that government assistance to the fishery, either in the 
harvesting or processing sectors, should be the exception rather than the rule. The fishery is a fully mature and, for the 
most part, fully exploited sector. Therefore, government subsidization would have the effect of modifying the 
decisions of those participating in the fishery — for instance, fishers might stay in the fishery even though it would not 
otherwise be in their economic interest. We observed that several jurisdictions have adopted a policy of subsidy 
elimination. 


Value added. We noted that promotion of the concept of highest possible value added provides a higher net economic 
return to the individual and the economy. In the end, it promotes a greater respect for the resource. 


Action and timeliness. The legal framework, including a conceptual understanding of sustainability, is very important 
but needs to be supported by appropriate, timely action to ensure that sustainability is actually achieved. 
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Collaborative Arrangements 


Issues for the Federal Government 


Main Points 


5.1 Collaborative arrangements are an alternative way — a potentially more innovative, cost-effective and 
efficient way — to deliver programs and services that traditionally have been provided by federal government 
departments and Crown corporations. In collaborative arrangements, the federal government, other levels of 
government and organizations in the private and voluntary sectors agree to share power and authority in decisions 
on program and service delivery. 


5.2 With the growing use of these arrangements, more taxpayer dollars are being spent and the risks need 
correspondingly more attention. The risks include arrangements set up poorly among the partners, limiting their 
chances for success; partners not meeting commitments; insufficient attention to protecting the public interest; 
insufficient transparency; and inadequate accountability. 


5.3 We believe that serving the public interest, effective accountability and greater transparency are basic 
elements of a framework for these arrangements, and we suggest questions that parliamentarians might wish to 
raise when assessing them. 


Background and other observations 


5.4 In the desire for greater efficiency, it is very important that the federal government and its partners not 
lose sight of the public purpose behind the collaborative arrangement, and of the need to provide transparent, fair 
and equitable service to the public. 


535 Effective accountability is more complex in a collaborative arrangement. The federal government is 
accountable to Parliament for the use of federal funds and authorities, to its partners for keeping its commitments, 
and, with its partners, to the public for the results the arrangement produces. In our view, this shared 
accountability means that more parties are accountable and it in no way lessens the federal government’s 
accountability for its own responsibilities in the arrangement. 


5.6 Delivering programs and services to the public through collaborative arrangements often requires more 
transparency than traditional delivery by a government department. Because partnerships are involved, it may be 
more difficult for citizens to know who is responsible. Consequently, the federal government needs to be as open 
as possible with information about agreements, decisions and results of the arrangements. 


Sy The Treasury Board Secretariat has informed us that it intends to continue to provide advice and to 
develop guidance on collaborative arrangements for federal departments and agencies that will address many of 
the issues identified in this study. 
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Introduction 


5.8 This study examines 
collaborative arrangements between the 
federal government, other levels of 
government and the private and voluntary 
sectors. 


59 Collaborative arrangements, also 
called partnering, are increasingly being 
used in federal program and service 
delivery as a management tool and to 
share power and authority with the 
government’s partners in making 
decisions. In our view, they have the 
potential to be an innovative, 
cost-effective and efficient way of 
delivering programs and services. 


5.10 Examples of collaborative 
arrangements include: 


e the Canada Infrastructure Works 
program; 


e Labour Market Development 
Agreements with the provinces and 
territories; and 


e The National Action Program on 
Climate Change and related initiatives. 


alll Another study in this Report, 
Chapter 6, examines two federal- 
provincial-territorial programs of what has 
become known as the “social union”. 
These programs, the National Child 
Benefit and Employability Assistance for 
People with Disabilities, illustrate many of 
the challenges of collaborative 
arrangements. In particular, they show the 
need for overall reporting when there is 
shared accountability, and the desirability 
of obtaining comparable and accurate data 
from the different partners. 


5.12 The Treasury Board has 
expressed the rationale for the use of a 
collaborative arrangement as an 
alternative to the traditional federal 
structure of departments, agencies and 
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Crown corporations in the following 
terms: 


The government will cooperate and 
develop partnering arrangements 
among departments and with other 
levels of government and other 
sectors of the economy. These 
arrangements will help it create new 
working relationships, exercise 
influence and leadership in the 
national interest, avoid costly 
duplication and overlap in services, 
and build on the strengths and 
capacity of other sectors to provide 
programs and services that are 
responsive to the client, innovative 
and affordable. (Framework for 
Alternative Program Delivery, 1995) 


S13 The government has stated: 


There are many alternatives to 


traditional departmental structures for 


delivering programs, and the 
government 1s vigorously pursuing 
those alternatives... Partnerships are 
an important form of alternative 


service delivery. Partnering with other 
governments, voluntary organizations 


and the private sector helps the 
federal government reduce overhead 
costs and duplication, and bring 
services closer to citizens. (Getting 
Government Right: Governing for 
Canadians, 1997) 


5.14 We undertook this study for 
several reasons: 


e More taxpayer dollars are being 
spent this way. Partly because of Program 
Review, the federal government has been 
making greater use of collaborative 
arrangements and has committed itself to 
doing still more. 


e There are risks that deserve 
attention. These include the risk of poorly 
defined arrangements, limiting the 
chances for success; partners not meeting 
commitments; insufficient attention to 
protecting the public interest; insufficient 
transparency; and inadequate 
accountability. 


e Many collaborative arrangements are 


new and not well understood. 


Collaborative 
arrangements have the 
potential to be 
innovative, 
cost-effective and 
efficient. 


As more taxpayer 
dollars are being spent 
through collaborative 
arrangements, there 
are risks that deserve 
attention. 
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Collaborative 
arrangements have 
common objectives 
tied to a public policy 
purpose, shared 
governance and 
written agreements on 
governance and 
financing. 


3.15 The choice of a collaborative 
arrangement as the means of program and 
service delivery is a policy decision of the 
federal government, which has 
consequences for Parliament and the 
Canadian public. In a collaborative 
arrangement, the federal government is 
one of a number of partners, and the 
partners operate in a governance regime 
where together they determine how key 
decisions will be made. As a result, 
accountability to ministers, to Parliament 
and to the public may become diffused 
unless care is taken in establishing and 
managing the arrangements. In contrast, a 
government department is directly 
accountable to the responsible minister, 
who answers for it in Parliament, and it 
operates within established rules and 
procedures for the allocation, control and 
management of human and financial 
resources. 


5.16 In our view, there is a need to 
better understand how to manage the risks 
associated with collaborative 
arrangements. There is also a need to 
know what questions to ask and what 
issues to consider when assessing these 
arrangements. 


What are collaborative arrangements? 


B17 For the purposes of this study, we 
are examining arrangements among 
autonomous organizations. These 
arrangements have the following features: 
common objectives tied to a public policy 
purpose, shared governance, and written 
agreements on governance and financing. 


5.18 Common objectives tied to a 
public policy purpose. The collaborative 
arrangement reflects an involvement of 
the federal government as well as other 
parties in the lives of Canadians, within 
the legal framework approved by 
Parliament. In this context, the federal 
government and its partners pursue 
common objectives and results that have a 
public policy purpose. 


1. 
oo 


5.19 Shared governance. The 
participating organizations share 
governance related to public policy as 
well as to the way in which the 
arrangement itself is governed. The 
process for making strategic decisions 
about collective activities is based on 
agreement by the participating 
organizations. They agree on the decisions 
that matter for the collaborative 
arrangement and that determine its future 
course of action. They also share the risks 
involved in those decisions. There is 
consultation among the organizations, so 
that decisions are not taken unilaterally. 
The way decision making is shared and to 
what extent will vary considerably with 
the type of arrangement. 


5.20 Written agreements on 
governance and financing. The 
organizations need to recognize the 
importance of agreeing on ways to steer 
the collective effort, as well as the 
importance of controlling the use of 
resources. A variety of mechanisms and 
types of agreements may be used. 


5.21 Arrangements excluded from 
the study. We have excluded the type of 
collaborative arrangement that involves 
only federal organizations. An example is 
the Federal Buildings Initiative, which 
Natural Resources Canada promotes to 
other federal departments to encourage 
comprehensive energy efficiency upgrades 
and retrofits in federal government 
facilities. While the issues facing 
collaborating federal organizations are 
significant and deserving of attention, they 
are not the same as those between the 
federal government and outside parties. 


5.22 Other arrangements not examined 
in this study include: 


e contracting for goods and services, 
and related circumstances where an entity 
outside the federal government acts as its 
agent (such contracting may occur within 
a collaborative arrangement; however, if 
the relationship is limited to that of a 
contractor or agent, the opportunities for 
collaboration are likely limited because 
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the federal government specifies the terms 
and conditions that must be met); 


e for the same reasons, traditional 
grants and contributions that do not 
involve collaboration; and 


e arm’s-length relationships — where 
organizations exercise independent 
authority to spend federal funds (for 
example, the Canada Foundation for 
Innovation). 


Focus of the study 


5.23 The objectives of the study were 
to examine the major issues related to 
effective participation in collaborative 
arrangements, identify desirable attributes 
of agreements and good implementation 
practices, and provide a framework for 
parliamentarians to assess the 
arrangements. A number of arrangements 
are presented as examples throughout the 
study; additional information about them 
can be found in the Appendix. Further 
details on the study are discussed at the 
end of the chapter in the section About 
the Study. 


Observations 


The Challenge of 
Collaborative 
Arrangements 


5.24 Building and maintaining 
collaborative arrangements is an 
alternative way of delivering government 
programs and services that requires the 
federal government to share program 
management and delivery with its 
partners. These arrangements require 
special effort and are not easily 
established or maintained. To a significant 
extent, they depend on the leadership 
shown by the key parties. 


Ensuring effective leadership 


525 The basis for a collaborative 
arrangement is the accomplishment of 


———————————— 
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mutually beneficial tasks. Each 
participating organization expects that the 
arrangement will lead to better client 
service or other benefits to Canadians than 
it alone could provide. To this end, the 
arrangement needs leadership to create a 
vision of where the partners want the 
arrangement to go and to translate that 
vision into reality. The nature of the 
leadership needed will depend on the 
nature of the collaborative arrangement, 
the tasks to be done and the roles and 
responsibilities of the partners. 


5.26 The identification of leadership 
roles may be a challenge. It should not be 
assumed that the federal government will 
always take the lead, particularly when 
other levels of government are involved or 
a number of partners are contributing 
equally. Rather, leadership needs to be 
based on expertise and level of 
involvement, and established through 
action, commitment and working 
co-operatively with other partners. 


S20 There are different levels and 
phases of leadership in a collaborative 
arrangement. Top-level political and 
bureaucratic leadership is often necessary 
to initiate an arrangement. These leaders 
need to demonstrate clear, consistent and 
visible commitment to the common 
objectives. Leadership at the level of 
program management is also essential. In 
addition, when more than one federal 
department is involved, a champion or 
leader in each department may be needed 
to take issues forward. Finally, to sustain 
the arrangement, buy-in and commitment 
are needed from front-line managers and 
staff. 


5.28 In some collaborative 
arrangements, the federal government 
does provide leadership. For example, in 
the National Action Program on Climate 
Change, the federal government indicated 
that it is committed to strong leadership to 
make sure that Canada stays on track to 
meet agreed targets (see Exhibit 5.1). In 
other cases, the commitment may be 
implied rather than stated explicitly. For 


The arrangement 
needs leadership to 
create a vision of 
where the partners 
want to go and to 
translate that vision 
into reality. 


Building and 
maintaining 
collaborative 
arrangements requires 
special effort that 
depends on the 
leadership shown by 
the key parties. 
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Collaborative 
arrangements need to 
overcome many 
obstacles. 


Exhibit 5.1 


Ensuring Effective Leadership 
in Climate Change 


instance, in initiating a program the 
federal government may assume a 
leadership role. 


Dealing with complex relationships 


5.29 Collaborative arrangements need 
to overcome many obstacles, ranging from 
the complexity of the activity to 
differences of view among the partners. 
Each participating organization pursues 
goals related to its own interests as well as 
the common goals of the arrangement. In 
the case of governmental partners, 
different legislative mandates also apply. 
Different approaches to managing people 
may also complicate the relationships 
among the partners. The genuine need for 
a collaborative arrangement must be 
strong enough to overcome these 
obstacles. 


5.30 Cases such as climate change and 
the North American Waterfowl 
Management Plan illustrate the 
complexity of collaborative arrangements. 
Both involve other levels of government 
and the treaty power of the federal 
government. Partners in collaborative 
arrangements may be disparate 
organizations, including various levels of 
government, First Nations and private 
corporations. An example is the Canadian 
Model Forest Program. 


Co-ordinating efforts among partners 
and within the federal government 


5.31 Experience suggests that 
establishing a co-ordination capacity 
makes the arrangement work better. A 
co-ordinator can be an organization, an 
individual or a body, such as a board or 
committee. The co-ordination capacity 
begins with a clear understanding of the 
autonomy of the participating 
organizations, thereby defining the scope 
for action for each party. Effective 
co-ordination then gradually improves 
performance by averting conflict, 
arbitrating differences and establishing 
priorities. It depends on good 
communication and on information 
systems aimed at voluntary compliance. 


5.32 We observed the use of 
secretariats as co-ordinators in a number 
of arrangements. An arm of a federal 
department may fulfil this role or, as in 
the Canada Infrastructure Works program, 
a joint federal-provincial body. Our 1996 
audit of Phase I of that program found that 
federal-provincial relations were 
particularly positive in cases where jointly 
funded secretariats had been established 
(see Exhibit 5.2). 


5:33 The background to federal 
participation generally includes related 
program activity in one or more 
departments. The respective roles and 
responsibilities of those departments are 


Canada is a party to international agreements on climate change, and the federal government is 
involved in arrangements with the provincial and territorial governments to implement these 
agreements in reducing greenhouse gas emissions. The two key federal departments involved are 
Environment Canada and Natural Resources Canada; many activities of both departments support 
climate change initiatives. We reported on an audit of climate change in the 1998 Report of the 
Commissioner of the Environment and Sustainable Development. 


Notwithstanding the federal government’s commitment to provide strong leadership on climate 
change, our audit identified, as a key element in managing climate change commitments, the need 
to specify the leadership roles and responsibilities of all federal players as well as all levels of 
government. We found that the management structure did not adequately specify federal 
responsibilities. This has contributed to an implementation gap, with performance falling short of 


expectations. 


The key federal departments report that since the release of our audit findings in May 1998, they 


are actively addressing these concerns. 
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sometimes not clear. The autonomy of a 
minister and his or her department from 
other ministers and departments may 
complicate an arrangement. Thus it is 
incumbent on those responsible for federal 
participation to be aware of related 
authorities and activities in other 
departments and to ensure a cohesive, 
consistent approach. 


Building trust and confidence 


5.34 The essence of collaboration is 
mutual trust and confidence among the 
parties to the arrangement. Little can be 
accomplished without it. Establishing trust 
and confidence depends on each party’s 
belief that the other parties have the skills 
and resources to do the job. It also 
depends on commitment and on the good 
faith demonstrated in making and 
implementing agreements. The parties to 
the arrangement need to consult fully with 
each other and share in decision making to 
ensure that their interests are considered 
and, where possible, protected. 


5.35 We observed different approaches 
to building trust. In the Canada 
Infrastructure Works program, the division 
of responsibilities between the federal and 
provincial governments was based on the 
comparative expertise of each level of 
government. This approach contributed to 
positive relations between them. 
Implementing the Canadian Industry 
Program for Energy Conservation requires 
confidence in the capacity of the private 
sector, since private sector voluntary 
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targets are used in place of other policy 
instruments to attain program objectives. 


A Framework for 
Assessing Collaborative 
Arrangements 


5.36 We believe that the federal 
government needs to manage the risks of 
participation in a collaborative 
arrangement by focussing on serving the 
public interest, effective accountability 
and greater transparency. 


e Serving the public interest. In 
providing programs and services to 
Canadians, there are always a number of 
parties whose interests are at play: the 
taxpayer, special interest groups, 
recipients of the programs and services 
and, in many collaborative arrangements, 
the private sector. As government and its 
partners pursue a public policy purpose, 
they need to balance competing interests 
and ensure that the public purpose and the 
need to provide transparent, fair and 
equitable service are not lost in the desire 
for greater efficiency. 


e Effective accountability 
arrangements. In a collaborative 
arrangement, the federal government is 
accountable to the Canadian Parliament 
for the use of federal public funds and 
authorities, to its partners for delivering 
on its commitments and, with its partners, 
to the public for the results achieved. 


e Greater transparency. 
Collaborative arrangements involving the 


This time-limited federal-provincial program was established in 1994 to assist in funding the 
maintenance and development of infrastructure, defined as physical capital assets instrumental in 
the provision of public services. The program involves a number of federal departments, with 

| central co-ordination from the Treasury Board Secretariat. We audited Phase I of the program in 


1996, and will report on Phase II later this year. 


Federal-provincial secretariats were used as one means to co-ordinate delivery of the program. Our 


| audit found that federal-provincial relations were particularly positive in Alberta and Manitoba, 
| where jointly funded secretariats had been established. Among other things, these secretariats 


operational databases. 


provided a focal point for contact and information, and permitted the development of joint 


The essence of 
collaboration is mutual 
trust and confidence 
among the partners. 


Exhibit 5.2 


Co-ordinating Efforts 
Among Partners - Canada 
Infrastructure Works 
Program, Phase | 
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Meeting the objective 
of greater efficiency 
should not 
compromise the 
objective of achieving 
the intended results. 


Exhibit 5.3 


Framework for Assessing 
Collaborative Arrangements - 
A Summary 


federal government need to be as open as 
possible with information on the 
agreements, decisions, objectives, targets 
and achievements. 


S37 For each of the three elements, 
we have identified what we believe are 
desirable attributes of agreements and 
good implementation practices, and have 
developed a series of questions that 
parliamentarians may wish to use as a 
framework for assessing collaborative 
arrangements. These questions are 
summarized in Exhibit 5.3. 


Serving the Public Interest 


5.38 Are objectives being met? The 
use of a collaborative arrangement is 
usually linked to a desire for better, more 
affordable federal program and service 
delivery. Over time, the arrangement 
should achieve its intended results. 


539 Is serving the public interest 
being given appropriate emphasis? A 
collaborative arrangement may enhance 
flexibility, cost effectiveness and citizen 
participation in program and service 
delivery. The federal government is 
constantly seeking more efficient ways to 


deliver programs and services, such as 
arrangements with the private sector and 
through “single windows” — the use of 
common facilities with provincial 
governments. However, meeting this 
objective of greater efficiency should not 
compromise the objective of achieving the 
intended results. In addition, given the 
complexity of collaborative arrangements, 
care has to be taken in enhancing 
accountability. 


5.40 The suggested elements of a 
workable collaborative arrangement — 
enhancing accountability, improving 
efficiency and achieving results — tend to 
pull in different directions. The emphasis 
in setting up an arrangement should not be 
solely on greater efficiency or on meeting 
accountability requirements. Through 
agreement with its partners, the federal 
government needs to keep these factors in 
balance, and transparently so, while 
giving priority to achieving the results of 
the arrangement (see Exhibit 5.4). 


5.41 Are public service values being 
maintained? Balancing the objectives of 
efficiency, effectiveness and 
accountability is not enough. In addition, 
the public interest needs to be kept in 


Serving the Public Interest 


e Are objectives being met? 


e Is the collaborative arrangement the best way to do it? 


e Is serving the public interest being given appropriate emphasis? 


e Are public service values being maintained? 


Effective Accountability Arrangements 


e Are the objectives, the expected level of performance and results and the operating conditions 


agreed to and clear? 


e Are the authorities, roles and responsibilities of each partner clear? 


e Are the expectations for each partner balanced with its capacities? 


e Can performance be measured and credibly reported to Parliament and the public? 


e Has adequate provision been made for review, program evaluation and audit? 


Greater Transparency 


e Have the information needs of those affected been recognized? 


e Is appropriate and sufficient information being disclosed to Parliament and the public? 
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view by assuring that services and benefits 
are delivered in a fair, impartial and 
equitable manner, in accordance with 
traditional public service values. 


Effective Accountability 
Arrangements 


5.42 In keeping with the principle of 
ministerial responsibility, a minister is 
accountable to Parliament for the 
involvement of a federal department or 
agency in a collaborative arrangement 
and, with partners, is accountable for the 
achievement of results. 


5.43 From our previous audits and 
studies of accountability arrangements, we 
have identified a set of key elements 
needed for strong accountability: 


e clear and agreed expectations; 

e clear roles and responsibilities; 

e balanced expectations and capacities; 
e credible reporting; and 


e reasonable review, program 
evaluation and audit. 


Clear and agreed expectations 


5.44 Agreement in writing on the 
common objectives and public policy 
purposes, as well as on the key results 
sought in the arrangement, is critical to its 
success and a necessary first step in its 


Enhancing Accountability 


Improving Efficiency 
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implementation. In addition, operational 
matters will need to be sorted out and 
agreed to. Although agreement on all 
these matters must ultimately be reached, 
not all of the specific details need to be 
finalized before beginning to implement 
the collaborative arrangement. 


5.45 A focus on the common 
outcomes being sought — particularly 
those seen as important to Canadians — 
rather than on specific outputs can be a 
powerful means of establishing and 
maintaining the collaboration. The efforts 
and steps taken to get the partners to agree 
on common objectives and results can be 
used to create understanding among the 
partners, sort out roles and 
responsibilities, encourage co-ordination, 
allow for flexibility in the specific 
contributions of different partners and 
build trust. 


5.46 Are the common objectives 
agreed to? The agreement should clearly 
describe the common objectives and 
purpose of the arrangement. Any 


statement of values, objectives, purpose or 


vision will serve as the basis for 
subsequent agreement on more specific 
results and performance measures. 
Without agreement on reasonably clear 
common objectives, the arrangement is 
unlikely to succeed. 


5.47 The parties may have different 
reasons for pursuing the common 
objectives. For example, in the North 
American Waterfowl Management Plan, 


Achieving Results 


A focus on common 
outcomes being 
sought can be a 
powerful means for 
establishing and 
maintaining 
collaboration. 


Exhibit 5.4 


Serving the Public Interest — 
Balancing Efficiency, 
Accountability and Results 


Source: Adapted and reprinted by permission of Harvard Business School Press. From Alliance Advantage : The Art of Creating Value 
through Partnering. Boston, MA 1998, pp 130. Copyright ©1998 by the President and Fellows of Harvard College, all rights reserved. 
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The partners need to 
agree on the results to 
be accomplished. 


Exhibit 5.5 


Agreement on Results — 
North American Waterfowl 
Management Plan 


duck hunters and bird watchers find 
common purpose in restoring waterfowl 
populations, albeit for very different 
reasons. 


5.48 Similarly, a private sector 
company pursuing a profit motive may 
find that its goal is compatible with the 
public interest, as expressed in the 
objectives sought through a collaborative 
arrangement. For example, the Canadian 
Industry Program for Energy Conservation 
encourages energy efficiency 
improvements that meet both public and 
private sector objectives. 


5.49 Initially, the parties may be able 
to agree only on a general statement of 
their common objectives, which may 
allow discussion on an arrangement to 
move forward. However, as they 
implement the arrangement and get to 
know each other, they need to reach 
agreement on more specific details 
concerning each partner’s role and the 
results to be accomplished. An example in 
which partners first reached agreement on 
general principles is the National Child 
Benefit, as discussed in Chapter 6 of this 
Report (paragraph 6.42). 


5.50 Are the expected results clear? 
In addition to agreeing on the common 
objectives, it is important that the partners 
agree on the results to be accomplished. 
The partners need to be as specific as 
possible about these results in the initial 
agreement and, if necessary, ensure that 
they are fully specified in subsequent 
discussions or agreements. 


5.51 To implement a collaborative 
arrangement, the responsible parties need 
to know what to do and what results are 
expected. A strategic framework is often 
required, elaborating on or clarifying these 
expectations and guiding activities. For 
more complex multi-year arrangements, a 
planning function is needed. 


5.52 The partners also need to agree 
on the performance measures that will be 
used to assess what has been 
accomplished. This will reduce the chance 
of major disagreement in the future on 
whether the arrangement has been 
successful. Performance measures can 
range from quite specific quantitative 
indicators to more general qualitative 
statements of what would have to occur 
for the arrangement to be judged a 
success. For example, the North American 
Waterfowl Management Plan clearly 
stated and communicated the expected 
results to all partners, and the related 
performance measures were agreed upon 
and then used in periodic reviews of 
results (see Exhibit 5.5). 


5.53 In addition, the partners need to 
agree on what data are required, key 
definitions, and how the data will be 
collected to ensure their integrity. 


5.54 Are the operating principles 
and procedures to be followed clear and 
agreed to? Each partner may be bound by 
legislative or administrative requirements 
related to certain aspects of the 
arrangement. For example, the federal 
government may want to make sure that 
specific decisions are made about the 


The North American Waterfowl Management Plan (NAWMP) is designed to restore waterfowl 
populations through a North American partnership of federal, state and provincial/territorial 
government agencies, non-government organizations, the private sector and landowners. The Plan 
involves the Canadian Wildlife Service of Environment Canada, and was examined in Chapter 11 
of our October 1997 Report (Moving toward Managing for Results). 


The NAWMP clearly stated and communicated the expected results to all partners, in the form of 
quantified targets of wetland habitat to be protected and waterfowl populations to be achieved. 
Related performance measures were agreed upon; that is, the annual count of the various species of 
ducks, geese, etc., and the size of the area of wetlands made suitable as waterfowl habitat. 
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delivery of services in both official 
languages, access to information and 
adherence to appropriate public service 
values such as impartiality, fairness and 
equity. The partners need to decide in 
advance on the extent to which any such 
requirements are to be met. 


5.55 Have human resource 
management issues been addressed? 
Managing people well is crucial to 
success. Various issues may arise when 
employees of participating organizations 
work together in a collaborative 
arrangement. There may be a need to have 
their assignments allow for career 
development, for example. Assignments 
also need to be structured to avoid a 
conflict of duties. Where a proposed 
collaboration arrangement has human 
resource implications, these should be 
addressed early and openly with all 
potentially affected employees and their 
representatives. 


5.56 In federal-provincial 
arrangements, employees of one level of 
government may report to those of 
another. Clarity of these relationships will 
be required. As well, federal employees 
continue to be subject to federal 
legislation and policies and to maintaining 
their rights and benefits in areas such as 
collective bargaining, grievance and 
redress, employment equity and official 
languages. 


aaa, As part of the Treasury Board 
Secretariat’s efforts to eliminate policy 
and legislative barriers, the federal 
government recently adopted a more 
flexible approach to human resource 
management in collaborative 
arrangements. This has been done through 
legislative changes that allow deputy 
ministers of departments to delegate the 
full range of human resource management 
authorities to non-federal managers. 


5.58 Has provision been made for 
adequate financial control? 
Collaborative arrangements generally use 


SS 
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federal public funds, public servants and 
other resources. The federal government is 
responsible for the stewardship of these 
resources. The other partners may also 
contribute significant resources, in some 
cases more than the federal government. 
Consequently, it is in the interest of all the 
partners to agree on sound procedures and 
practices for financial management and 
control, dealing with such things as 
different accounting methods and year-end 
carry-OVver provisions. 


Shoe In addition, agreements need to 
identify and minimize the risk of the 
federal government being held liable for 
amounts exceeding parliamentary 
appropriations. This can be done by 
explicitly documenting the mechanisms 
and conditions under which any losses 
would be shared or guaranteed, and by 
capping maximum federal exposure at 
levels of approved authorities. 


5.60 In striking an agreement that 
provides for adequate financial 
management systems and processes, the 
federal government needs to work with 
the other partners toward an appropriate 
standard of financial management, 
tailored to the nature and complexity of 
the arrangement. 


5.61 Phase I of the Canada 
Infrastructure Works program is an 
example of the arrangements that the 
federal government can make in 
agreement with its partners. In our audit of 
the program, we examined the basis on 
which federal officials had approved 
projects; we found that there was a need to 
improve financial control. In the program 
as a whole, insufficient provision had been 
made for compliance audits. However, 
federal program officials reached an 
agreement with the Quebec government 
department involved in the arrangement to 
set in place a system of compliance audits. 
In our judgment, this contributed to 
meeting the federal government’s 
requirement for assurance of compliance 
with essential conditions of the program. 
Systems and process audits and reviews 
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Managing people well 
is crucial to success. 


It is in the interest of 
all the partners to 
agree on sound 
procedures and 
practices for financial 
management and 
control. 
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Clear roles and 
responsibilities are 
essential for 
accountability. 


Exhibit 5.6 


Shared Accountability 


were also undertaken in the four Western 
provinces. 


Clear roles and responsibilities 


5.62 Are agreed roles and 
responsibilities clear? Clear roles and 
responsibilities in written agreements need 
to be established from the outset and to 
reflect the interests and capabilities of 
each partner, as well as the common 
interests. They are essential to 
accountability. Accountability 
relationships in collaborative 
arrangements are sometimes referred to as 
“shared accountability”. This means that 
the partners are collectively accountable 


for the success and operation of the 
arrangement, and hence share 
accountability (see Exhibit 5.6). In our 
view, shared accountability means only 
that more parties are accountable and in 
no way lessens the federal government’s 
accountability for its own responsibilities 
in the arrangement. 


5.63 The involvement of federal and 
other government partners may be based 
on legislation, which may limit their 
actions and activities. The legal, human 
resource and financial authorities need to 
be clearly identified, including the sharing 
or transfer of human, physical, financial or 
technological resources. Collaborative 


Collaborative arrangements involve several partners who work together toward some shared 
common objectives. These partners are collectively accountable for its success and operation, and 
hence share accountability. Without care, however, this may result in a diffusion of accountability 
due to a lack of clarity about responsibilities and action taken. 


In collaborative arrangements, there are three kinds of accountability relationships: 


e accountability among the partners; 


e accountability between each partner and its own governing body, such as, in the federal case, 


ministers and Parliament; and 


e accountability to the public. 


A collaborative arrangement involves an obligation of each partner to the other partners to fulfil 
and report back on its own part of the arrangement. Mechanisms are needed to ensure that this 
occurs and also that the collectivity can credibly report as a whole on its shared accomplishments. 


The creation of the partnering arrangement does not reduce the accountability of the federal 
government to Parliament for the use of federal funds and authorities. Since the accountability 
relationships are more complex, the federal partner needs to ensure that the arrangement is 
structured so that there is an appropriate level of reporting back to ministers and Parliament. 


Moreover, the federal government must take responsibility for managing toward the intended 
results through its actions and decisions in the arrangement. 


In addition, the accountability elements of the arrangement must include provisions to ensure that 
the federal government can credibly demonstrate in a timely manner: 


e the extent to which the objectives and expected results of the collaborative arrangement are 
being achieved; 


e areasonable assessment of the federal contribution to those results, namely to what extent it has 
made a difference; and 


e what has been learned through the arrangement. 


Hence, in a shared accountability arrangement the federal partner still has quite specific 
responsibilities and requirements, as do all the partners. To be able to account for the elements 
described above, the federal partner is responsible for organizing and managing the relationships 
with its partners so that it can obtain necessary information, monitor results and make (or require) 
adjustments as needed. 
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arrangements may also involve the 
delegation of authorities and the transfer 
of resources from the federal government 
to other (usually governmental) partners 
or to individuals or bodies representing 
the collectivity. In such cases, the nature 
of the transfer needs to be clearly 
specified and publicly disclosed. 


5.64 Labour Market Development 
Agreements illustrate the accountability 
relationships in collaborative 
arrangements involving the federal 
government with other levels of 
government (see Exhibit 5.7). 


5.65 Other collaborative arrangements 
use contribution agreements as a financing 
mechanism. While these agreements may 
include management and co-ordination 
provisions, their purpose is to ensure that 
payment of federal funds is conditional on 
performance and on compliance with the 
program. There is a need to also provide 
federal departments in collaborative 
arrangements with guidance on how to use 
contribution agreements to manage the 
sharing of power and authority in decision 
making. For example, in our 1996 audit of 
Phase I of the Canada Infrastructure 
Works program, which is a contributions 
program, we found that federal-provincial 
agreements contained management and 
co-ordination provisions but did not 
clearly identify roles and responsibilities 
for providing monitoring and performance 
information. Consequently, the federal 
government did not have adequate 
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information for accountability and 
program implementation. 


5.66 Have adequate decision-making 
processes been established? The extent 
to which decision rules need to be 
formally specified in the agreement will 
depend on the nature of the arrangement. 
For diverse organizations, a wide range of 
rules may be required, encompassing 
membership, participation, the make-up of 
decision-making bodies and the conduct 
of business. Where the parties are closely 
aligned, the decision process may rely on 
a more informal but still clear 
understanding. In the case of the federal 
government, the roles and responsibilities 
of ministers, or senior public officials 
acting for ministers, need to be clearly 
delineated. 


5.67 As noted in paragraph 5.33, 
effective co-ordination is needed in order 
to clarify roles and responsibilities when 
multiple departments are involved. The 
solution may lie in designating a lead 
department, the involvement of a central 
agency or establishing a specific 
management structure. 


5.68 Through the management 
structure, decision rules are implemented, 
the necessary steering and working 
committees are established, and the 
involvement of ministers, senior public 
servants and those appointed as officers of 
the arrangement 1s specified. The 
management structure of the arrangement 
should fit with the management structures 
of the participating organizations and 


labour market services to the public. 


reporting on results. 
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Labour Market Development Agreements, signed or under negotiation with all of the provinces 
and both territories, involve various forms of shared delivery of federal, provincial and territorial 


Some jurisdictions have chosen co-management with the federal government; others have opted to 
receive full transfer of federal responsibilities. The Agreements include accountability provisions, 
with respect to objectives, expected results, performance measurement, monitoring and reporting. 
For example, the Canada-Alberta Labour Market Development Agreement establishes measures 
(indicators) and targets for expected results, and requirements for program evaluation and 


Exhibit 5.7 


Accountability Relationships - 
Labour Market Development 
Agreements 
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Adequate provision for 
reporting of overall 
results is also 
essential for 
accountability. 


Exhibit 5.8 


Decision Making, Financial 
Control and Review - 
Canadian Model Forest 
Program 


include the mechanisms needed to 
co-ordinate efforts and resolve problems. 


5.69 We observed a variety of 
management structures, including boards 
of directors and management committees 
supported by subcommittees and 
sometimes a secretariat. In some cases, 
the parties adopt a constitution or charter 
— for example, in the Canadian Model 
Forest Program (see Exhibit 5.8). In other 
cases, terms of reference specify the roles 
of committees and related 
decision-making structures. In our audits 
of climate change and Phase I of the 
Canada Infrastructure Works program, we 
found that the management structures did 
not adequately specify federal 
responsibilities. 


Balanced expectations and capacities 


5.70 Have the partners the 
capability to do what they expect? The 
partners form expectations about what 
each will do, and what they will do 
together, to advance the common agenda. 
A balance needs to be struck between 
these expectations and the capacity of 
each partner to deliver. The partners need 
to assess each other’s competence and 


experience. Expectations should be based 
on an accurate appreciation of capacities, 
including authorities, skills and resources. 
In some cases, the abilities and capacities 
of those delivering key elements of a 
program or service may need to be 
strengthened and the additional cost of 
this recognized. 


5.71 The National Child Benefit 
provides an example of the need for such 
a balance, as discussed in Chapter 6 of 
this Report (paragraph 6.59). If the 
smaller partners in the National Child 
Benefit lack the capacity to obtain 
accurate and relevant data on program 
outcomes, and to verify the data, then the 
federal government or larger provinces 
could work with the partners, if asked, to 
help them build capacity. 


Credible reporting 


S372 Adequate provision for reporting 
in agreements, leading to credible and 
timely reporting of plans, activities and 
overall results, is essential for 
accountability in a collaborative 
arrangement. 


Ch) Is provision made for adequate 
reporting? In a collaborative 
arrangement, the partners report to each 


The Canadian Model Forest Program is a national network of large-scale working models of 
sustainable forest management. These models are designed to expand the range of forest uses and 
the benefits, in keeping with the principles of sustainable development. The Canadian Forest 
Service of Natural Resources Canada established the Program in 1992. We audited it in 1993 and 


did a follow-up in 1995. 


In the Eastern Ontario Model Forest (EOMBP), for example, an agreement, a constitution and 
by-laws define the roles and responsibilities of the federal government and the other partners. The 
constitution sets out a management structure, including the appointment and powers of a board of 


directors and officers. 


Each model forest produces an annual report. Control over public funds is exercised through a 
requirement for audited annual financial statements, oversight of expenditures by the board, and 
provisions for progress reports and inspection-of records in subagreements between the particular 


model forest and other partners. 


Review and adjustment occurs through the board’s exercise of its responsibility to monitor 
progress and report on corrective measures, and through a requirement for program evaluation. A 
strategic plan reported on the accomplishments of the first phase of the EOMEF and was used to set 


out goals and strategies for the next phase. 
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other and to their respective governing 
bodies, both individually and collectively; 
they also report to those affected by the 
arrangement. In an agreement, a statement 
of the reporting required is essential for 
effective accountability to the governing 
bodies, as well as for management and 
co-ordination purposes. The agreement 
should normally refer to such reporting 
vehicles as business plans, corporate 
reports, or reports to Parliament and to 
provincial and territorial legislatures. 


5.74 We observed many forms of 
reporting in collaborative arrangements, 
including annual reports issued by 
organizations outside the federal 
government. In some cases, these reports 
provide performance information. Where 
collaborative arrangements have been 
established to move program and service 
delivery closer to those directly affected, 
reporting to the public and to the 
community fulfils an accountability 
obligation. We suggest that the reporting 
regime ought to serve accountability to 
the community affected, to the applicable 
minister(s) and, through the minister(s), to 
Parliament. It is also important that the 
federal government work with its partners 
to ensure that the information needed for 
good reporting is provided on a timely 
basis, so that decision makers can make 
effective use of it. 


S15 Is information for Parliament 
sufficient? We noted in a number of cases 
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that reporting by collaborative 
arrangements is not addressed or 
referenced in the established performance 
reporting systems of government, 
particularly those for reporting to 
Parliament. For example, private sector 
partners may make an annual report public 
and provide it to the minister who answers 
in Parliament for the federal department 
involved. This practice is followed for the 
annual report of the Canadian Industry 
Program for Energy Conservation (see 
Exhibit 5.9). To ensure that all such 
reports are brought to the attention of 
Parliament, we believe that they need to 
be clearly referenced in the responsible 
department’s Estimates documents. These 
documents are the Reports on Plans and 
Priorities and departmental Performance 
Reports, and there are established 
procedures allowing parliamentary 
committees to examine them. 


5.76 Is the reporting credible? In 
Chapter 5 of our April 1997 Report, we 
suggested a number of desirable features 
for performance reports on what has been 
accomplished: 


e Clear context and strategies. The 
report should clearly describe the mission 
and mandate of the organization, the 
objectives of its programs and services, 
the major strategies and resources being 
used to achieve these objectives and the 
related external context. 


e Meaningful performance 
expectations. The report should contain 


voluntary initiative of Canada’s manufacturing and mining industries aimed at reducing energy 
use. Natural Resources Canada developed a partnership with CIPEC to encourage energy 
efficiency improvements through voluntary programs in industry. We reported on our audit of 
energy efficiency measures in that Department in our April 1997 Report. 


| The Canadian Industry Program for Energy Conservation (CIPEC), established in 1975, is a 


CIPEC co-ordinates the development of energy efficiency goals (annual targets), action plans and 
services for each industrial sector through a network of task forces. A task force council works in 
| close co-ordination with Natural Resources Canada, and is supported by a secretariat in the 


Department. 


_ An annual report contains information on improvements in energy efficiency, based on a system 
| developed to monitor and report on industry’s progress toward its targets. This report is provided 


to the Minister of Natural Resources. 


Exhibit 5.9 


Decision Making and 
Reporting - Canadian Industry 
Program for Energy 
Conservation 
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The partners need to 
agree on a mechanism 
to monitor progress 
and evaluate 
performance and 
results. 


clear and concrete key performance 
expectations with a focus on 
outcome-results. 


e Results reported against 
expectations. Key results should be 
reported in relation to expectations and be 
attributable to the activities undertaken. 


e Demonstrated capacity to learn 
and adapt. The report should provide a 
measure of the ability of the program to 
learn from past performance and to adapt 
to external changes. 


e Fair and reliable performance 
information. Performance information 
should be fair, valid, reliable and 
understandable. 


Bir In collaborative arrangements, 
credible performance reporting will often 
depend on information that the partners 
provide to each other. Partners will need 
to both seek assurance that the 

information provided by others is based on 
good data and provide such assurances 
themselves. 


5.78 External audit assurance can 
enhance the credibility of reports on 
results by examining the consistency, 
reliability and fairness of the partners’ 
performance information. 


Reasonable review, program evaluation 
and audit 


SE ku Has provision been made for 
reasonable internal audit and program 
evaluation? The partners need to agree on 
a mechanism to monitor the progress of 
collective activities and evaluate 
performance and results. The federal 
government’s use of internal audit and 
program evaluation may constitute such a 
mechanism. Requirements for audit and 
evaluation, including their reporting, can 
be set out in contribution agreements and 
memoranda of understanding between 
partners in the collaborative arrangement. 


5.80 As we noted in our May 1996 
Report Chapter 3 on program evaluation, 
evaluation studies have the potential to: 


e provide information to support 
decisions about resource allocation; 


e help Canadians determine the value 
obtained with their tax dollars; and 


e help public servants and their 
collaborative partners manage for results 
and take responsibility for them. 


5.81. For example, as discussed in 
Chapter 6 of this Report (paragraph 6.105), 
the partners in the Employability 
Assistance for People with Disabilities 
have agreed to co-ordinate the planning 
process and to evaluate program results. 


5.82 Is sufficient monitoring under 
way? Monitoring is increasingly 
recognized as an integral part of sound 
management. Program managers gather 
and analyze information that will allow 
them to gauge progress toward objectives 
and adjust program planning and 
implementation. In a collaborative 
arrangement, a realistic and effective 
monitoring strategy is required that takes 
into account: 


e the nature of the arrangement, such 
as the historical relationship between the 
partners, the level of innovation involved 
and the complexity of the arrangement; 


e the level of political or financial risk 
associated with the program and the 
arrangement; 


e the specific accountability 
requirements for each partner; 


e the capacity of the partners to 
monitor performance; 


e the type and level of delegation to 
each partner (for example, if program 
delivery has been delegated to a partner, 
monitoring also can be delegated. The . 
federal government could work with its 
partners to ensure that the monitoring 
takes place and results are reviewed); and 
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e the monitoring already undertaken 
by the partners (as much as possible, 
monitoring systems and information 
should be shared to avoid duplication and 
promote efficiencies). 


5.83 Is sufficient learning taking 
place? The accountability relationship is 
meant to be a positive one that the 
participating organizations can use to 
improve performance — their own as well 
as that of the arrangement as a whole. For 
instance, if the agreement specifies 
mechanisms to review performance and 
calls for institutional learning, the partners 
have to make this work in practice. If 
expectations are clearly not being met, 
corrective actions need to be taken. The 
co-ordination process needs to 
accommodate such adjustments. 


5.84 Are procedures in place to 
follow if things go wrong? It is important 
that agreements include specific 
procedures, such as dispute resolution 
mechanisms, that each party can follow 
should things go wrong. These procedures 
may include reference to independent 
bodies to assess cases of disagreement, 
and ultimately should provide ways to 
terminate the arrangement if necessary. 
Provisions for termination also need to 
ensure that public funds and assets, 
including the sale of any public assets, are 
safeguarded. 


5.85 Has provision been made for 
audit? The parties need to agree on 
appropriate provisions for external audit 
of the arrangement. A requirement for 
financial audit is often recognized in 
collaborative arrangements, but given that 
there are public purposes involved, 
value-for-money audits and independent 
assessment of performance information 
may be needed. The question of who 
should do the audits also needs to be 
determined. In the case of 
federal-provincial arrangements, this 
Office recognizes the value of working 
jointly with provincial audit offices in 
audits of program delivery. 
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Greater Transparency 


5.86 It is important that government 
ensure appropriate transparency in the 
delivery of services to the public. 
Delivering services by collaborative 
arrangements often requires greater 
transparency than traditional delivery by 
government departments. There are 
several reasons for this: 


e The involvement of several different 
partners may make it difficult for citizens 
to know who is responsible for doing 
what, unless extra care is taken to 
communicate such information. 


e Where citizens now have to deal 
with service providers outside 
government, access to redress mechanisms 
may not be evident. Citizens may not have 
the same recourse to the minister and 
members of Parliament in these cases. 


e The provisions of the Access to 
Information Act that apply to the federal 
government may not extend to partnering 
organizations, limiting the information 
made available unless the arrangement 
specifically addresses the issue. The 
partners may need to reconcile access to 
information and privacy laws of different 
jurisdictions. 


5.87 Are the information needs of 
the affected public recognized? 
Collaborative arrangements delivering 
public programs often provide information 
about their activities to the communities 
directly affected as well as to the general 
public. This form of transparency 
complements the responsibility of the 
federal government and the other partners 
to make adequate information available to 
Parliament and the public. For example, 
the Labour Market Development 
Agreements and reports are available for 
public scrutiny. 


5.88 Is sufficient information being 
disclosed to Parliament and the public? 
Information about a collaborative 
arrangement ought to be made available to 
Parliament, the public, and the respective 
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The use of 
collaborative 
arrangements to 
deliver services to the 
public often requires 
greater transparency 
than traditional means. 


Information about 
collaborative 
arrangements ought to 
be made available to 
Parliament and the 
public. 
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legislative and governance bodies, with 
the exception of sensitive and personal 
information. 


5.89 In addition to the agreements 
themselves and the reports they produce, 
the information made public should 
include: 


e major decisions and activities that 
affect specific groups; 


e related documents such as non- 
commercially confidential board or 
management committee papers; and 


e statements of policies and practices, 
encompassing conflict of interest rules, 
limits on confidentiality and complaint 
and redress mechanisms. 


5.90 It is important that a 
collaborative arrangement not be used to 
reduce transparency. The principles 
supporting the intent of the Access to 
Information Act need to be recognized in 
setting up a collaborative arrangement. 
The responsible parties are pursuing a 
public purpose and therefore have a 
responsibility to be as open as possible 
about their decisions and actions. They 
should give reasons for their decisions and 
restrict information only when necessary. 


5.91 Part of the commitment to greater 
transparency is consultation with the 
public. It is important to ensure that the 
goals sought and the results obtained in a 
collaborative arrangement are the right 
ones for Canadians. To this end, the 
partners need to seek out the views of 
target groups, stakeholders and the general 
public. 


5.92 Successful collaborative 
arrangements also depend on transparent 
relations among partners. A wide range of 
information on such aspects as goals, 
reporting relationships, resources and the 
results being achieved needs to be made 
available among the partners, to build and 
maintain trust and to enable the necessary 
co-ordination to take place. 


Conclusion 


5.93 As more taxpayer dollars are 
being spent through collaborative 
arrangements, the associated risks require 
greater attention. These include the risk of 
poorly defined arrangements, 
commitments not met, insufficient 
attention to protecting the public interest, 
insufficient transparency and inadequate 
accountability. 


5.94 —_ In recognition of the need to 
manage these risks, this study examined a 
number of issues related to effective 
participation in collaborative 
arrangements. In serving the public 
interest, the federal government must 
ensure that its public purpose is not lost in 
the desire for greater efficiency. Effective 
accountability means that the government 
remains accountable to Parliament for the 
use of public funds and authorities but 
also shares with its partners accountability 
to the public for results. Transparency 
calls for the federal government to be as 
open as possible about the arrangements it 
enters into with partners. 


5.95 The study identified a number of 
desirable attributes of agreements in a 
collaborative arrangement. The partners 
need to be very clear about the results 
expected for the common objectives, 
about roles and responsibilities, and about 
provisions for reporting, evaluation and 
audit. Operating procedures also need to 
be set out, including those for human 
resource management and financial 
control. 


5.96 It is important that these 
attributes of agreements be complemented 
by good implementation practices. In 
addition to setting out provisions in an 
agreement for reporting, the federal 
government and its partners need to 
ensure that performance reports credibly 
describe the results accomplished and 
attribute them to the resources used and 
the actions taken. 
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5.97 This chapter points to the need 
for federal departments to effectively 
manage their involvement in collaborative 
arrangements. The Treasury Board 
Secretariat and departments have an 
important role in providing advice and 
guidance on these management issues. In 
addition, there is a need for all of those 
involved in collaborative arrangements to 
learn from the experience. For our part, 
we plan to report on an audit of federal 
government involvement in collaborative 
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arrangements and new arm’s length 
relationships. 


5.98 The desirable attributes of 
agreements and the good implementation 
practices our study identified led to a 
series of questions that we suggest as a 
framework parliamentarians may wish to 
use to assess collaborative arrangements. 
As curther experience is gained with 
collaborative arrangements, we expect to 
be able to refine this framework. 
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ty About the Study 


Objectives 


The objectives of the study were to examine the major issues related to effective participation in collaborative 
arrangements, identify desirable attributes of agreements and good implementation practices, and provide a 
framework for parliamentarians to assess the arrangements. As experience is gained with collaborative 
arrangements, we expect to refine and improve this framework. 


Scope 


To carry out this study, we drew upon related audits and studies carried out by this Office, reviewed the 
relevant academic literature and conducted selected casework relating to federal involvement with other 
levels of government, the private sector and other organizations. We also made use of the following 
documents: 


e Modernizing Accountability Practices in the Public Sector, 1998 — a joint paper by Treasury Board 
Secretariat and Office of the Auditor General of Canada. 
(http://www.tbs-sct.gc.ca/rma/accoun/OAGTBS_E.html) 


e Assessing Alternative Service Delivery Arrangements, June 1998 — an Office of the Auditor General of 
Canada discussion paper provided to the Standing Committee on Public Accounts. 
(http://www.oag-bvg.gc.ca/domino/other.nsf/html/dispr_e.html) 


In this study, we did not compare practices against a predetermined set of criteria. Rather, we identified a 
series of questions that parliamentarians may wish to use as a framework for assessing collaborative 
arrangements. 


Study Team 


Assistant Auditor General: Maria Barrados 
Principal: John Mayne 
Project Leader: Tom Wileman 


Allison Fader 
Frances Smith 


For information, please contact John Mayne. 
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National Child Benefit and 
Employability Assistance for People 
with Disabilities 


Main Points 


National Child Benefit 


6.1 The National Child Benefit (NCB) represents a new form of collaborative arrangement between 
provinces (except Quebec) and territories and the federal government. A key challenge for all involved is to assure 
taxpayers that moneys are spent for the purposes intended, with due regard to economy and efficiency and with 
appropriate means to measure and report on effectiveness. Because there are many governments involved, it is 
also necessary to respect the jurisdictional competence of the different parties. As the NCB is implemented it is 
critical that, at a minimum, there be no less accountability because it is shared than if only one jurisdiction were 
involved. 


Background and other observations 


6.2 The goal of the National Child Benefit is to reduce the depth of poverty among families with children and 
to increase parental attachment to the work force. The NCB involves no new law, contract, or contribution 
agreement, but rather an increased federal child tax benefit (the NCB supplement) for low-income families — 
about $850 million in 1998, rising to $1.7 billion by 2000. In turn, jurisdictions providing social assistance 
benefits to families may reduce their payments by the amount of the increased tax benefit. They have agreed to 
reinvest these savings in programs that have mutually agreed-upon objectives and that benefit poor families with 
children. 


6.3 One of the clear achievements of the negotiations leading to the NCB was the partners’ agreement on the 
overall goals of both the federal and the provincial elements of the program. They also have committed to a new 
kind of joint accountability to the public. What is distinctive about this commitment is that no level of government 
is more responsible for reporting on the results than another. Each is accountable for the overall program. 


6.4 It will be a significant challenge in the first few years, before evaluation results are available, to 
demonstrate precisely how the National Child Benefit has contributed to reduced depth of poverty and increased 
employment among its recipients. In assessing progress, the NCB accountability report will also need to be clear 
about any trade-offs among goals and to discuss the implications. Otherwise, readers may expect that all goals can 
be achieved at the same time. 


6.5 There are potential gaps in the quality of financial and other information. The credibility of the 
accountability reports depends on the comparability, accuracy and verifiability of information from all parties — 
not only those who negotiated the arrangement (provinces, territories and the federal government) but also those 
who are responsible for designing and implementing only specific sub-programs (Ontario municipalities and First 
Nations). This is also a challenge for audit offices that serve the partners in this arrangement. 
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6.6 We think the key ingredients in reporting the NCB’s results to the public are transparency and adequacy 
of the information. This means: 


¢ appropriate care is taken to ensure that the information is credible and, at least for financial 
information, subject to audit; 


e data are adequate to determine if the program’s overall goals are being achieved; and 
e data are sufficiently comparable that outcomes of different provincial approaches can be compared. 


It must also be clear whose responsibility it is to do all this, and who will provide assurance that it has been done. 
Moreover, those charged with the responsibility must have the capacity to undertake the task. 


The Department has said that this chapter will be of considerable assistance in its work to build an 
accountability regime for the NCB. Some issues are already being addressed. The remaining issues identified 
in this study will be raised by the federal government in its capacity as co-chair of the federal-provincial- 
territorial group working on the NCB. 


Employability Assistance for People with Disabilities 


6.7 Employability Assistance for People with Disabilities (EAPD) is the other program to emerge from 
federal-provincial discussions in 1997-98 on the social union. The aim of this 50-50 cost-shared arrangement is to 
help people with disabilities overcome the barriers they face in the labour force. EAPD supports provincial and 
territorial programs and services ranging from programs that assist in the first steps toward developing skills to 
those that support a person at work so the person can keep working. Our interest in the program at this early 
juncture is that steps be taken to ensure that the information each partner will provide on its own expenditures and 
programs is credible, and permits comparisons of different approaches and assessments of the overall program’s 
effectiveness. 


Background and other observations 


6.8 Each province has signed an agreement with the federal government. Taken together, the agreements 
illustrate several of the elements of a collaborative arrangement discussed in Chapter 5 of this Report. For 
instance, the governments agree to follow a co-ordinated, participative planning process and to evaluate program 
results. 


6.9 Human Resources Development Canada (HRDC) and provincial partners have set about to solve 
implementation issues in a collaborative fashion. For example, the provinces have committed to annual 
accountability to HRDC. However, so far there is no specific commitment to the format, substance, or timing of 
any overall annual report, nor are there specific goals or targets over any time frame. We recommend that HRDC 
prepare a brief overall annual report for EAPD, comparing activities, expenditures, program outputs and 
performance of the partners. 


6.10 The partners still have important work to do. When so many jurisdictions are involved, those with 
oversight and audit responsibilities face the challenge of helping to ensure that the quality of financial and 
performance reporting is maintained and enhanced. 


The Department has said that the case study will be useful in its work with provinces. It agrees in principle 
with the recommendation to produce an overall annual report on EAPD expenditures and performance. It 
pointed out that it will be important to note that EAPD is only one part of overall programming for people 
with disabilities. 
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Social Programs 


Preface 


The federal government’s role in Canada’s social programs has undergone important 
changes over the last few years. Recently, it has agreed with nine provinces and two 
territories on a framework for new types of arrangements in a social union. Citizens, 
legislative bodies and, thus, audit offices may justifiably seek assurance that these new 
arrangements increase, or at least do not diminish, accountability for expenditures on 
shared social programs and their outcomes. 


This chapter looks at the challenges for accountability that characterize the two 
recently implemented social programs that are already part of the social union: the 
National Child Benefit and Employability Assistance for People with Disabilities. 


The case studies 


6.11 The National Child Benefit 
(NCB) is an innovative arrangement that 
combines federal tax expenditures and 
provincial programs. The aim is to reduce 
the depth of poverty in families with 
children, with a particular emphasis on 
providing incentives for low-income 
parents to enter and remain in the work 
force. Provincial premiers and the Prime 
Minister have referred to the NCB as a 
positive example of how social programs 
can be delivered collaboratively in a 
social union. 


6.12 Employability Assistance for 
Persons with Disabilities (EAPD) is also 
cited as an example of a social union 
program. It is a relatively more traditional 
form of federal cost-sharing in 
provincially delivered programs. Its aim is 
to help people with disabilities overcome 
the barriers they face in the work force by 
helping them to prepare for, obtain and 
maintain employment. 


Challenges for accountability are great 


6.13 Accountability has been a key 
element for designers of both programs. In 
official statements, all partners have made 
a commitment to account for the 
expenditure of funds and for the 
outcomes. In the case of the National 
Child Benefit, territories, provinces 
(except Quebec) and the federal 


government have committed to an annual 
joint accountability report and, eventually, 
to evaluations of the program. This is the 
first joint report of its kind, due in the 
spring of 1999. 


6.14 For EAPD, provinces have 
committed in bilateral agreements to 
publish reports annually starting in 
1999-2000. They have also agreed to 
share and make public the results of 
evaluations and to conduct a review of the 
arrangement at the end of its third year, in 
2001. 


6.15 Notwithstanding these 
commitments, it is not clear exactly how 
accountability for overall results of these 
programs will be achieved. An important 
question for this Office is, How should the 
Auditor General, in fulfilling his 
obligations to Parliament, look at 
programs that are not fully federal? Our 
Office clearly has an interest in 
arrangements that use federal resources. 
Parliament expects us to cover such 
arrangements in our work and to report on 
them (see Appendix A). Since all levels of 
government agree on the need to be 
accountable to citizens and legislatures for 
these programs, our task is simply to assist 
in this endeavour. The case studies in this 
chapter show that there is no single best 
approach to demonstrating accountability, 
but there are some basic principles that 
can usefully be followed. 


All levels of 
government have 
committed to 
accountability but the 
question is, How will 
this be accomplished? 
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Social Programs 


The initiatives 
emerged from 
collaborations among 
provincial premiers. 


By 1997, the National 
Child Benefit was seen 
as a model for future 
social programs in 
Canada. 


Canada’s approach to social programs 
is changing 


6.16 Both the National Child Benefit 
and Employability Assistance for People 
with Disabilities can best be understood in 
the context of the ongoing 
federal-provincial collaboration that 
produced them. Although they are now 
referred to as examples of the “social 
union”, they were negotiated separately 
and not as part of any overall framework. 
The initiatives emerged from several 
interprovincial and federal-provincial- 
territorial meetings, conferences, councils, 
working groups and committees. Although 
decisions taken in such forums do not 
have the force of law, they impact directly 
on social program design and expenditure. 


6.17 Annual Premiers’ Conferences. 
Each summer, premiers meet at Annual 
Premiers’ Conferences to discuss matters 
of mutual interest without federal 
government participation. At the 1995 
Premiers’ Conference, after major cuts in 
federal transfers to provinces had been 
announced, premiers reiterated their desire 
to work together on national matters 
affecting areas of provincial jurisdiction 
and to speak with a common voice in the 
national debate on social policy reform. 


6.18 Ministerial social policy 
councils. In 1995, premiers formed a 
Ministerial Council on Social Policy 
Renewal to deal specifically with social 
policy issues. At the 1996 Premiers’ 
Conference, they invited the federal 
government to join them in establishing a 
reconstituted Federal-Provincial-Terri- 
torial Council on Social Policy Renewal to 
advance the agenda of collaboration. This 
Council oversees social policy initiatives 
involving more than one department. In 
November 1996, it agreed that its first 
priorities were an integrated child benefit 
and improved services to persons with 
disabilities. The overall goals of 
co-operation were to reduce duplication 
and renew the social safety net. 
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6.19 First Ministers’ Meetings. All 
premiers and the Prime Minister attend 
First Ministers’ Meetings. In 1996, First 
Ministers began to discuss a collaborative 
and accountable approach to Canada’s 
social programs. In communiqués, they 
referred to the National Child Benefit as 
part of a broader National Children’s 
Agenda. They cited child poverty and 
persons with disabilities as collective 
priorities. The 1997 Speech from the 
Throne referred to the agreement among 
all levels of government to “work together 
to develop this broader agenda for 
children, including clear outcome 
measures by which to gauge success.” 


6.20 At the First Ministers’ Meeting in 
December 1997, they reiterated the need 
to negotiate “a set of principles for social 
policy, such as mobility and monitoring 
social policy outcomes.” They affirmed 
the National Child Benefit as a possible 
model for future social programs: the 
federal government would increase the 
Canada Child Tax Benefit, while 
provinces would use their resulting 
savings in welfare expenditures to 
undertake programs consistent with 
mutually agreed-upon goals. They also 
requested that social services ministers 
conclude the development of a vision 
statement and national framework for 
assistance to persons with disabilities. 


6.21 Ministers Responsible for 
Social Services. After the 1996 First 
Ministers’ Meeting, the task of program 
design for the National Child Benefit and 
Employability Assistance for People with 
Disabilities had been assigned to Ministers 
Responsible for Social Services. This 
group had many of the same members as 
the Council on Social Policy Renewal. In 
face to face meetings and through 
electronic communication, 
federal-provincial-territorial ministers and 
their public servants set about to design 
the NCB and EAPD. 


6.22 Working groups of public 
servants. Officials were organized into 
the NCB Working Group and the EAPD 
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Working Group. Each group was 
co-chaired by a provincial and a federal 
official. There were also sub-groups to 
work on specific issues. While Human 
Resources Development Canada (HRDC) 


had lead responsibility at the federal level, 


the Department of Finance, Revenue 
Canada, Indian and Northern Affairs 
Canada and Health Canada were all 
represented. The groups reported to 
Ministers Responsible for Social Services 
and to the Council on Social Policy 
Renewal. Each member also reported to 
his or her superiors and minister. 


6.23 Public consultation. Most of the 
ministerial meetings also had a public 
component: the decisions that ministers 
wished to make public were announced 
through press releases, interviews, and 
postings to the Internet 
(http://socialunion.gc.ca/menu_e.html). 
As well, the groups and their ministers 
received input from a variety of formal 
and informal consultations with experts, 
advocates for the poor and the disabled, 
and other non-governmental 
organizations. 


Quebec’s policies are consistent, even if 
not formally aligned 


6.24 Quebec’s general position has 
been that all federal funding for social 
programs should be transferred directly to 
Quebec. It has nevertheless negotiated an 
agreement on EAPD. For the NCB, 
Quebec wanted to develop its program 
proposals according to its own priorities 
and timetable. Notwithstanding its 
concerns, Quebec representatives have 
participated as observers at ministerial 
meetings that discussed both programs, at 
the Council on Social Policy Renewal and 


on most of the working groups of officials. 


Quebec residents receive the Canada 
Child Tax Benefit on the same terms as all 
other residents of Canada. As well, 


Human Resources Development Canada — Accountability for Shared 
Social Programs 


Quebec has introduced several programs 
for children that would very likely have 
qualified as part of the NCB program. 
Indeed, programs for children in Quebec 
are generally aimed at reducing child 
poverty and increasing incentives for 
greater parental attachment to the work 
force, the same aims as the National Child 
Benefit’s. The most significant difference 
is that as long as Quebec remains a 
non-participant, its programs for children 
will not be included in the joint 


accountability report on the National A major challenge for 


Child Benefit. 
both shared 
arrangements is how 
Focus of the study to achieve credible 


reporting of overall 
results. 


6.25 The two programs we studied 
represent a situation where a federal 
department is involved in program 
delivery but does not have sole 
responsibility, and where new kinds of 
accountability arrangements define 
relationships between governments. Our 
purpose in studying the NCB and EAPD 
was to arrive at answers to the following 
questions: 


e How can credible reporting on these 
two programs best be achieved? 


e What are realistic expectations for 
measuring the program outcomes? 


e Are there information gaps that will 


need to be filled? 
Our study was 


undertaken at an early 
stage to help achieve 


6.26 In approaching these questions, 
we focus primarily on the accountability 
reports that will be published annually for 


each program. It is through these reports appropriate 
that readers will be able to assess the 
programs and how well the federal arrangements. 


government and its partners have fulfilled 
their joint commitment to accountability. 
The case studies also illustrate and draw 
on Chapter 5 of this Report, where we 
suggest a framework for parliamentarians 
to use in assessing collaborative 
arrangements. 
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Social Programs 


6.27 Appropriate accountability 
arrangements must include the provision 
of relevant, verifiable information to 
parliamentarians and others to assess 
whether funds have been spent for 
approved purposes and expected outcomes 
have been achieved. Providing such 
information when many partners are 
involved is not an easy task. We believe 
that making some general suggestions 
while the accountability arrangements are 
still being implemented will help achieve 
appropriate arrangements in a timely 
fashion. 


6.28 Further details on our approach to 
the case studies can be found in About the 
Study at the end of the chapter. Appendix 
A contains information on our earlier 
work that addressed related issues in 
HRDC, more general issues of 
performance measurement and the quality 
of provincial accountability information in 
a federal-provincial context. Appendix B 
provides historical context for the NCB. 
Appendix C compares the two programs. 
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The National Child Benefit 


Introduction 


6.29 The National Child Benefit 
(NCB) combines two significant trends of 
the last 20 years — the trend toward 
delivery of social programs through tax 
benefits and the trend toward shared 
initiatives negotiated with provinces and 
territories. It is innovative in that federal 
tax benefits and provincial programs 
together aim to achieve mutual objectives. 


6.30 Another important feature of the 
NCB is that all participants have 
committed to joint public reporting on the 
program and on the extent to which it is 
achieving its intended outcomes. What is 
distinctive about this commitment is that 
no level of government is more 
responsible than another for reporting on 
the results. All are responsible for the 
overall product. We believe that this is the 
first time this kind of co-operative effort 
has been undertaken in Canada. 


6.31 The case study sought to: 


e help develop an awareness of what 
has been achieved so far with the National 
Child Benefit, and of the factors that may 
delay or constrain the establishment of 
appropriate accountability arrangements 
and credible measurement of results; and 


e assist the reader of the annual 
“public accountability reports” by 
discussing what can reasonably be 
expected from them. 


NCB supplement: The federal 
contribution 


6.32 In the first phase of the NCB, the 
federal government’s annual financial 
commitment is $850 million beginning in 
1998. This includes $250 million 
announced in the 1996 Budget and 

$600 million announced in the 1997 
Budget to supplement the Canada Child 
Tax Benefit (CCTB). In the 1998 Budget, 


Eee 
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the federal government announced that it 
would provide an additional $425 million 
in 1999 and another $425 million in 2000, 
bringing its total contribution to 

$1.7 billion per year beginning in 
2000-01. This commitment was 
reaffirmed in the 1999 Budget. 


6.33 The federal contribution to the 
NCB is an estimate based on economic 
models and Revenue Canada data. The 
amounts of the NCB supplement that will 
actually be paid depend on family 
incomes in the previous year. We did not 
verify the models, and it is too early to 
know if they have correctly anticipated 
total levels of benefits. Variances could 
also occur because of economic and 
labour market changes. 


6.34 Although millions of non-poor 
families receive the base Canada Child 
Tax Benefit, only low-income families 
benefit from the NCB supplement. This 
supplement is being introduced over 
several years. In 1997 the Working 
Income Supplement benefit to working 
poor families was restructured from a flat 
$500 per family to a per child basis: $605 
for the first child, an additional $405 for 
the second child and $330 more for each 
additional child. In 1998 this became the 
NCB supplement to all families with 
income under $20,922. The supplement 
increased the base CCTB to a maximum 
of $1,625 for one child, $3,050 for two 
and $4,475 for three. 


6.35 However, as a tax benefit 
recipients could also receive less than the 
maximum amount. For each dollar of net 
income over $20,921, the maximum NCB 
supplement is phased out by 12.1 percent 
for a one-child family, 20.2 percent for a 
two-child family and 26.8 percent for 
larger families, reducing to zero when 
income reaches $25,922. The 1999 Budget 
maintained approximately the same rates 


Social Programs 


The combination of 
federal tax benefits 
and provincial 
programs is 
innovative. 


Joint accountability 
reports to the public 
on outcomes and 
expenditures are a key 
feature. 
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Social Programs 


Provinces and 
territories have agreed 
to “reinvest” savings 
from reduced social 
assistance payments 
in new programs that 
are consistent with the 
objectives of the 
National Child Benefit 
(NCB). 


of phase-out when it increased the 
maximum NCB supplement by an 
additional $180 per child in 1999 and a 
further $170 per child in 2000. With a 
larger benefit to be phased out, the cut-off 
point at which the supplement is reduced 
to zero will increase to $27,750 in 1999 
and $29,590 in 2000. 


6.36 The 1997 federal Budget 
indicated (see Appendix B) that although 
the intent of the NCB was to reduce social 
assistance by the amount of the 
supplement, a family on social assistance 
would be no worse off as a result of the 
NCB. However, the discontinued Working 
Income Supplement was not counted as 
income, whereas the NCB supplement was 
to be considered income and deducted 
dollar for dollar from social assistance 
benefits in most jurisdictions. Therefore, 
some families on social assistance who 
had received the Working Income 
Supplement in 1997 would potentially 
receive less in 1998. To avoid this, 
provinces and the federal government 
agreed to share 1998-99 costs of 
transitional assistance (under provincial 
social assistance budgets) for families 
whose income would otherwise have 
declined. It is not known whether this 
arrangement will continue beyond 

June 1999. 


Reinvestments: The contributions of 
provincial, territorial, municipal and 
First Nations governments 


6.37 The financial commitment of 
provinces, territories, First Nations and 
Ontario municipalities is equal to the 
savings they will realize in reducing social 
assistance payments by the amount of the 
NCB supplement. The participants in the 
negotiations agreed that these 
“reinvestments” would be used to develop 
“programs and benefits reflective of each 
jurisdiction’s special needs and priorities. 
These reinvestments are consistent with 
the goals of the NCB: to reduce the depth 
of child poverty and help low-income 


families find and keep work.” These could 
include: 


e income support programs and tax 
measures for families with children and 
with incomes below certain levels; 


e earnings supplements for families 
with children and with employment 
income in a specified range; 


e extension to working poor families 
of in-kind benefits available for children 
in families receiving social assistance (for 
example, health benefits); 


e improvement in overall child 
benefits to families receiving social 
assistance; 


e social services, such as child care, 
that support parents’ attachment to the 
work force; and 


e other initiatives aimed at preventing 
and reducing child poverty, such as child 
nutrition and teen parent programs. 


6.38 Specific child care subsidies and 
initiatives, for example, have been 
introduced or enhanced as part of the NCB 
in Newfoundland, New Brunswick, Prince 
Edward Island, Nova Scotia, Manitoba, 
and Alberta. Earned-income or child 
support supplements for families with 
children have been introduced or 
enhanced in Ontario, British Columbia, 
Saskatchewan, Nova Scotia and the 
Northwest Territories. Alberta, 
Saskatchewan and Prince Edward Island 
have introduced health benefits. 


6.39 Not all provinces reduced social 
assistance payments by the full amount of 
the NCB supplement. Newfoundland and 
New Brunswick chose to maintain social 
assistance payments at their previous 
levels, and thus had no savings to reinvest. 
Nevertheless, each instituted additional 
programs aimed at low-income families: 
New Brunswick allocated a portion of 
what it would have saved, and 
Newfoundland the entire amount. 
Although Alberta initially reduced social 
assistance payments by the amount of the 
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NCB supplement, it found that it was 
spending less than anticipated on its main 
reinvestment initiative, a child health 
benefit. It therefore increased shelter 
benefits for families on social assistance 
who have children. 


6.40 In June 1998, provinces (except 
Quebec) and territories announced the 
level of their commitments to the NCB: a 
total of $329.4 million between July 1998 
and June 1999. (These commitments will 
increase in July 1999 as the NCB 
supplement increases.) The First Nations 
commitment is $27.8 million for the same 
period. 


Observations 


NCB Goals Involve Trade-offs and 
Performance Measurement 
Challenges 


6.41 In negotiations, it is often helpful 
to agree first on general principles. 
However, this is not the end of the 
process. As any negotiator knows, “the 
devil is in the details.” 


6.42 The partners negotiated three 
objectives for the National Child Benefit: 


e reduce the depth of child poverty; 


e increase parents’ attachment to the 
work force; and 


e reduce administrative overlap and 
duplication. 


6.43 It is important to understand how 
these goals interact with each other in 
order to appropriately assess the 
program’s success. The underlying 
premise is that in the long term, child 
poverty will be reduced if parents on 
social assistance enter the work force and 
remain in it. The strategy of the National 
Child Benefit is to reduce child poverty by 
providing income supplements and other 
assistance to working families. The 
intention is to increase the benefit over 
time until it is large enough to eliminate 
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the “welfare wall” and is “sufficient to 
remove benefits for children from the 
welfare system”. In other words, the 
long-term aim is to eliminate the 
economic disincentives to work that occur 
when social assistance benefits for 
families with children are more than the 
earnings and other benefits of a 
minimum-wage job. The expectation is 
that more parents will enter or remain in 
the work force and that their children will 
benefit from their parents’ improved 
status. These are all testable propositions 
and the challenge for reporting on the 
NCB will be to show that this is indeed 
what has happened. 


Reducing the depth of poverty 


6.44 The first goal of the National 
Child Benefit is to reduce the depth 
(rather than the incidence) of poverty. 
Depth of poverty is the amount by which 
family income is below a poverty line; 
incidence is the number of people living 
below that line. According to those 
involved in developing the NCB, depth of 


poverty is a better measure than incidence. 


An aim to reduce the incidence of poverty 
would simply encourage serving the 
“richest” of the poor, since it might focus 
on helping those nearest the line to move 
just over it. In aiming to reduce the depth 
of poverty, however, the program 
designers aim to assist families regardless 
of income: each additional dollar of 
income reduces the depth of poverty, 
independent of whether the family is close 
to or far from the poverty line. 


6.45 One key question for assessing 
the success of the NCB will be, How deep 
is the depth of poverty? As defined in the 
preceding paragraph, depth of poverty is 
the amount by which family income is 
below a poverty line. Using Statistics 
Canada’s low-income cut-off as a poverty 
line, HRDC estimates that in 1996 the 
aggregate depth of poverty for all families 
with children living below that line was 
$7 billion. (The total depth of poverty is 


Social Programs 


As any negotiator 
knows, “the devil is in 
the details”. 


The NCB goals involve 
testable propositions. 
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Social Programs 


The choice of a 
poverty line depends 
on political and social 
values, not just 
technical 
considerations. 


Even with the NCB 
supplement, some 
families may be 
economically less 
well-off working than 
on social assistance. 


Although the NCB 
emphasizes 
accountability to the 
public, accountability 
to Parliament and 
accountability of 
government to 
government are also 
important. 


greater because it also includes persons 
without children.) 


6.46 A second key question will be, 
How much has the NCB reduced the depth 
of poverty? The federal commitment for 
the NCB supplement to the base Canada 
Child Tax Benefit is $850 million 
beginning in July 1998, rising to 

$1.7 billion beginning in July 2000. The 
depth of poverty could theoretically be 
reduced by the entire amount if the value 
of all the provincial benefits were 
included. (Some provincial NCB 
programs, aimed at lowering the welfare 
wall by improving health benefits or 
subsidizing child care, might improve the 
economic situation of poor working 
families without providing direct income.) 
The depth of poverty could also be 
reduced by more than the supplement 
alone by counting the additional amounts 
earned by parents who enter the work 
force as a direct result of the NCB. 


6.47 The choice of poverty measure 
has an impact on measures of program 
performance. The resources required to 
meet any specific target for reducing the 
depth of poverty and to measure the 
degree of program success for a given 
level of expenditure will depend on how 
poverty is defined. A higher poverty line 
will mean that the aggregate depth of 
poverty is greater and therefore that more 
resources are required to reduce it. 
Ministers have decided to use two 
measures of low income for the first NCB 
report: Statistics Canada’s low-income 
cut-offs (LICOs) and the low-income 
measure (LIM). While this Office and 
other credible bodies may comment on 
overall criteria, data quality and other 
implementation and administrative issues, 
how and where a poverty line is 
constructed depends in the final analysis 
on fundamental social and political value 
judgments, not just technical 
considerations. 


Increasing parents’ attachment to the 
work force 


6.48 The premise for this goal is that 
in the longer term, working parents will be 
better able to meet the needs of their 
children than non-working parents. While 
some research supports this view, another 
view is that much more research is needed 
— especially on the outcomes of very 
young children whose parents work 
compared with those whose parents stay at 
home. Whatever the research concludes, 
NCB partners face a challenge in 
attributing changes in work force 
attachment to the NCB: first, changes in 
the economy unrelated to the NCB can 
affect labour markets; and second, at 
current NCB levels the benefits in most 
provinces reduce, but do not eliminate, the 
welfare wall. 


Reducing overlap and duplication 


6.49 Reducing overlap and duplication 
is an administrative goal. The challenge of 
achieving it is significant because the 
NCB can operate more efficiently if 
provincial social assistance data and 
federal child tax data are shared. There are 
problems of systems integration, and 
privacy laws may limit partners’ ability to 
share data for administrative purposes. 
The most obvious opportunities for 
administrative savings are in tax benefit 
programs that use Revenue Canada 
systems. 


There are trade-offs in the short term 
between two goals 


6.50 While goals are sometimes 
mutually reinforcing, in other cases one 
may be achieved at the cost of not fully 
achieving another. In the NCB, the 
long-term intent is that increasing parental 
attachment to the labour market reinforces 
the goal of reducing the depth of poverty. 
However, there are potential trade-offs 
between these goals in the short term. 
That is, people leaving social assistance to 
take low-wage employment may actually 
be worse off economically in the short 
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term, even with the NCB supplement and 
other NCB program benefits. This 
happens whenever the total value of 
wages and increased benefits attributable 
to the NCB is less than the value of social 
assistance payments and benefits. 


6.51 In their accountability reports on 
the NCB, the partners will need to clearly 
discuss any such trade-offs and their 
implications. Otherwise, readers of the 
reports may expect that all the goals can 
be achieved simultaneously. 


The goals are general, without specific 
targets or time frames 


6.52 Qualitative measures can be very 
useful indices to demonstrate the 
effectiveness of an arrangement. However, 
performance expectations should have a 
quantitative component whenever it is 
feasible and appropriate. The goals for the 
NCB do not contain quantifiable 
expectations of performance. For 
example, there is no specific target 
amount by which the depth of poverty is 
to be reduced. As the goal is stated, it 
could be achieved if the depth of poverty 
were reduced by one dollar. Reducing the 
depth of child poverty is a goal for which 
targets can feasibly be set. It should be 
possible to identify specific areas and 
targets for the reduction of duplication and 
overlap. If these goals had been expressed 
as quantitative targets with specific time 
frames, it would be easier to determine the 
degree to which governments have 
achieved their aims. 


The relationship of goals to 
performance expectations can be better 
explained 


6.53 Transparency of the assumptions 
and evidence that underpin the design of a 
social program is essential to assessing the 
program’s performance and outcomes. It 
is important for readers of the 
accountability report to know why the 
program is attempting to do what it is 
doing and what outcomes are anticipated. 
It is also useful to differentiate between 
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what is expected in the short term and 
what might be observed in the longer 
term. 


6.54 In the NCB, many of these 
relationships are made clear. For example, 
in the September 1997 background 
booklet, “Building a Better Future for 
Canadian Children”, good use is made of 
research demonstrating the importance of 
early childhood intervention (see 
Appendix B). Evidence that parental 
attachment to the work force contributes 
to better outcomes for children was not 
presented, perhaps because research is not 
yet conclusive on this issue (as we note in 
paragraph 6.48). The issue is very 
complex. Recent HRDC-sponsored 
research tends to support the conclusion 
that work force attachment does not 
negatively affect outcomes for children. 
Positive outcomes are correlated with 
education and with parents who read to 
their children, and less positive outcomes 
with being on welfare. However, it is not 
easy to sort out cause and effect so that 
social policy can be designed. This is why 
HRDC continues to sponsor such research. 


Accountability, Results 
Measurement, Audit and 
Evaluation Issues 


6.55 An essential feature of the NCB 
is that provincial and territorial 
governments are responsible for delivering 
programs that lie within the NCB 
framework. However, the level of funds 
they commit to reinvestments is 
contingent on the level of savings they 
achieve in reducing social assistance 
payments. Who is responsible for assuring 
that this commitment is met? This raises a 
key question of roles and responsibilities. 


Who is accountable to whom? 


6.56 The NCB Governance and 
Accountability Framework, released in 
March 1998, distinguishes 
government-to-government accountability 
from government accountability to 
legislatures and from government 
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Readers of the 
accountability report 
should be told whether 
the data are 
comparable, accurate 
and, at least for 
financial information, 
subject to audit. 


Each partner is 
responsible for its own 
data, and for obtaining 
assurance about the 
quality of the data that 
other partners submit. 


accountability to the public. For the NCB, 
“governments will emphasize 
accountability to the public.” However, in 
our view there is no need to emphasize 
one type of accountability over another. 
Accountability to the public for the NCB 
is consistent with, but different from, 
accountability to other governments, or 
accountability to legislatures. For 
example, accountability to Parliament is 
related to its annual approval of budgets 
and Estimates, whereas accountability to 
the public is related to the public’s role as 
electors of MPs. These differences are 
important. Care will be required to ensure 
that all three types of accountability are 
maintained. 


6.57 A key mechanism for 
demonstrating accountability is credible 
reporting. While mechanisms for reporting 
to the public tend to vary according to the 
nature of each program, arrangements for 
reporting to legislatures are well defined 
and involve auditors general and 
legislative committees. The partners in the 
NCB state that one advantage of 
accountability to the public is that it will 
“minimize administrative reporting.” 
However, accountability is unlikely to be 
served best by minimized administrative 
reporting. Administrative data are part of 
the accountability and operational 
relationship between partners. They are 
also necessary for reporting both outputs 
and performance outcomes. The goal, 
therefore, should be to report 
appropriately. At the same time, if 
accountability to the public means that 
performance reports on the overall 
program present the necessary relevant, 
credible and auditable information to the 
public rather than to internal 
administrations, this would be one 
appropriate way to deal with the NCB’s 
interjurisdictional aspects. 


Key Accountability Ingredient: 
Adequate Information 


6.58 In the NCB Governance and 
Accountability Framework, partners have 


agreed on the importance of obtaining 
relevant information. “The partners 
support the importance of evaluative and 
analytical work to ensure the appropriate 
and reliable measurement of program 
outcomes. The partners recognize the 
importance of ensuring that data provided 
for purposes of public reporting [are] open 
to public scrutiny.” Implementation needs 
to carry through on this commitment. 


6.59 However, smaller provinces and 
perhaps First Nations and municipalities 
may lack the resources to obtain accurate, 
comparable and relevant data and to 
verify them. This is an issue that can be 
addressed in the same collaborative spirit 
as has prevailed in the NCB design and 
implementation to date. For example, if 
asked, the federal government or larger 
provinces might work with partners to 
help them build capacity. 


6.60 Data do not have to be perfect, 
but they should be as good as possible. 
One important reason is that in 
negotiations it is important for the parties 
to have accurate information about their 
own and their partners’ contributions. This 
information 1s also useful for operational 
purposes. Later on, in reporting results 
and expenditures, data from several 
jurisdictions and levels of government will 
have to be aggregated and synthesized to 
assess whether progress is being made. 
The key ingredient to credible reporting, 
therefore, is that each partner take steps to 
ensure that: 


e the information presented is credible 
and, at least for financial information, 
subject to audit; 


e data are adequate to determine if 
overall program goals are being achieved; 
and 


e data are sufficiently comparable that 
outcomes of different provincial 
approaches can be compared. 


We expect that in negotiating agreements 
and, afterward, in reporting on the 
program, the federal government and its 
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partners will know and communicate 
information about the accuracy of the data 
they have received. 


6.61 The reinvestments raise 
particularly challenging questions of 
compliance, comparability and 
verifiability of data, and audit assurance. 
Clearly, each partner is responsible for 
data on its own contributions and, where 
assurance is required, can arrange for its 
own auditor to verify the information. 
Another possibility is that the partners’ 
auditors can co-ordinate this task 
themselves. Whatever arrangement is 
finally arrived at, each partner is 
responsible for obtaining assurance from 
other partners that all the information is 
accurate and relevant. It will be important 
that readers know what information has 
been verified or is consistent with 
independent analysis, what has been or 
will be audited or otherwise independently 
verified, and what information must be 
accepted on trust. 


6.62 We examined information made 
available to federal officials about 
provincial reinvestments during 1998, as 
the NCB was being implemented. 
Although we did not perform a formal 
audit, we do have some preliminary 
observations. 


Reinvestments are within the 
parameters of the NCB goals and 
objectives but there are information 


gaps 


6.63 According to the information that 
we have seen on reinvestments, their 
objectives appear to correspond clearly 
with at least one of the NCB objectives. 
However, there are gaps in the 
information. In general, the partners 
believe that most of these will be filled as 
the program matures. 


6.64 First Nations and Ontario 
municipalities will have funds available 
for reinvestments, but did not participate 
in the negotiations or the design of the 
accountability framework. As of February 
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1999, they were still formulating plans 
and had not announced them to the 
general public. (According to Indian and 
Northern Affairs Canada, some First 
Nations reinvestment programs have been 
announced to their own communities.) We 
do not yet know what measures they, like 
the provinces and territories, will take to 
ensure that their information is of 
adequate quality. 


6.65 With respect to First Nations 
programs, Indian and Northern Affairs 
Canada has informed us that it believes 
the steps taken to date will fulfil the 
obligation to account for the performance 
of the NCB in First Nations communities. 
It points out that since the reporting 
required from First Nations is based on the 
fiscal year, it will not receive information 
on the operations of reinvestment 
programs until July 1999. Indian and 
Northern Affairs believes that it will then 
be in a better position to assess the quality 
of data and the capacity of First Nations to 
provide it. It has said it is committed to 
working in partnership with First Nations 
to assess and evaluate the long-term 
impacts of the NCB on reserves. 


6.66 First Nations directly administer 
at the community level an estimated 

$592 million in social assistance 
programs. Indian and Northern Affairs has 
indicated that the First Nations 
commitment for on-reserve reinvestments 
in the first full year is $27.8 million — the 
fourth-largest reinvestment amount after 
Ontario, Quebec, and British Columbia. 
First Nations communities are very 
diverse in size, population, 
socio-economic conditions, governance 
and capacity. Indian and Northern Affairs 
points out that the introduction of a 
largely federal-provincial-territorial 
program into this diverse population, with 
concomitant reporting and accountability 
expectations, presents unique challenges. 
It has informed us that its regional offices 
have been working with First Nations and, 
where appropriate, with provinces and 
territories to develop mutually acceptable 
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So far, there is little 
information on exactly 
what reinvestments 
Ontario municipalities 
and First Nations are 
making in the NCB. 


It is easier to 
determine whether 
a certain amount 
of money has been 
spent than to 
determine whether 
in the longer term 
the expenditures 
are achieving 
program goals. 
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Social Programs 


The accountability 
report should state the 
level of actual 
expenditures and 
compare them with the 
commitments made by 
each partner. 


frameworks that will guide the 
implementation, funding and reporting 
requirements of the NCB reinvestment 
initiatives. Its aim is to have frameworks 
that are both consistent with the objectives 
of the NCB and flexible enough to address 
the different priorities and needs of First 
Nations communities. 


6.67 Because municipalities pay 

20 percent of social assistance 
expenditures, Ontario has asked them to 
design their own NCB reinvestment 
programs. Those will account for 

20 percent of Ontario’s reinvestments. At 
December 1998, municipalities had only 
begun to design their programs. The 
province has informed us that it is putting 
in place a requirement that municipalities 
provide it with the necessary information 
to demonstrate that they have met their 
obligations under the NCB. It indicated 
that it is a high priority for Ontario to have 
accurate and timely information on 
municipal strategies, expenditures, and 
program performance and it does not 
anticipate any gaps in information on 
municipal reinvestments. 


6.68 Finally, there is no information 
about Quebec’s reinvestments, since 
Quebec has participated in the discussions 
only as an observer. As we indicated in 
paragraph 6.24, Quebec has instituted 
programs that would likely have qualified 
as reinvestments. HRDC estimates that if 
Quebec had been a party to the agreement, 
its reinvestment envelope would have 
been approximately $150 million. 


Are reinvestments ‘‘new’’ money? 


6.69 The reinvestments not only must 
meet the objectives of the National Child 
Benefit but also must represent new 
programs or enhancements to existing 
ones. According to information supplied 
by the provinces, the reinvestments 
announced so far meet both these criteria. 
However, because that information has not 
been verified, there is no independent 
assurance yet that this is indeed the case. 


Definitive answers may come later; these 
programs are in the early stages of 
implementation. 


Are the data on the size of 
reinvestments reasonably accurate and 
is the degree of inaccuracy, if any, 
known and acceptable? 


6.70 Participating provinces and 
territories have agreed that the amounts 
they reinvest will be at least equal to the 
savings they realize in reducing welfare 
payments. Clearly, future savings can only 
be estimated. The amounts of the actual 
savings will have to be reported when they 
have been realized. As well, when there 
are uncertainties about the data it will be 
important to give readers information on 
the range of uncertainty, much as is done 
with polling results. However, it is also 
important to keep in perspective the 
question of whether the reinvestments are 
at the right levels. It is easier to determine 
whether a certain amount of money has 
been spent than to determine whether in 
the longer term the expenditures are 
achieving program goals. 


6.71 Estimating the amounts to be 
reinvested is not a straightforward task. 
For example, they depend on the number 
of children in families receiving social 
assistance, not on total population. Thus, 
Alberta plans to reinvest less per capita 
than Saskatchewan because it has fewer 
children in families on social assistance. 
Another complication is that reinvestment 
levels are related to the previous year’s 
income of families on social assistance, 
because this is how the amount of a 
parent’s Canada Child Tax Benefit is 
determined. Since families whose 
previous year’s income was over $25,921 
do not receive the NCB supplement, a 
province would not be entitled to reduce 
their social assistance. If the economy 
were to take a sudden downturn and the 
number of people on social assistance 
increased, the commitment to 
reinvestment would increase. However, 
the calculation would have to take into 
account that some of the new social 
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assistance recipients might have had too 
high an income the previous year to 
qualify for the NCB supplement, and for 
these individuals there would be no 
savings to reinvest. 


6.72 Federal officials have told us they 
are satisfied that provincial estimates of 
planned reinvestments are accurate and 
that correct levels of reinvestment are 
planned. We have not verified these 
calculations, and would need more 
information to do so. We have not seen 
estimates of an acceptable range of 
accuracy for the projections of provincial 
reinvestments, nor have we seen plans to 
verify the actual levels. We have no 
reason to believe that the calculations are 
inaccurate or that planned and actual 
expenditures will be less than agreed, 
except perhaps in the first year (see 
paragraph 6.80). 


Are there known data gaps and 
problems and are plans in place to fix 
them? 


6.73 A technical sub-group of the 
NCB Working Group has devoted a great 
deal of study to these issues. From the 
information made available to us, it is 
evident that many data issues were 
identified but, given the short time 
available for implementation, only the 
issues most critical to starting the program 
were addressed. 


6.74 One of the key remaining issues 
is how to obtain and link data on 
employment, social assistance, income tax 
and family structure without 
compromising privacy. For example, a 
two-way “data exchange” between 
provinces or municipalities and Revenue 
Canada could help reconcile differences in 
data on family structure, movement of 
children between caregivers, etc., so that 
overpayments and underpayments could 
be dealt with quickly and accurately, 
perhaps automatically. To enhance 
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comparability and increase efficiency, 
there may be a need to reconcile 
differences between data systems of the 
different jurisdictions. A related question 
will be whether data will be available for 
evaluation and analysis. 


A key issue is how to 
obtain and link data 
held by different 
partners without 
compromising privacy. 


Means to monitor and inform all 
participants and stakeholders 


6.75 Co-ordination and negotiation of 
the next phase of the NCB and reporting 
on its performance will continue. Given 
the collaboration that has taken place so 
far with provinces and territories, we 
would expect the federal government, as a 
partner, to be informed on a timely basis 
about changes to programs and levels of 
expenditure as well as outcomes. 
However, it is too early to be able to 
report that this is actually happening. 


What are the means for dispute 
settlement? 


6.76 The NCB is not a contract or 
contribution agreement. The NCB 
Governance and Accountability 
Framework offers partners several 
processes for resolving issues. These 
range from submitting the issues to bodies 
such as Ministers Responsible for Social 
Services to ad hoc bilateral processes, 
whose results are communicated to the 
other partners. However, in the final 
analysis there is no mechanism other than 
mutual agreement for settling disputes, 
and no provision for sanctions if the 
agreement is not observed. The federal 
government has no direct means to 
influence provincial actions since the 
Canada Child Tax Benefit goes directly to 
families, not provinces. Similarly, despite 
the high degree of collaboration, it is 
ultimately the federal government that 
decides the level and basic structure of the 
tax benefit. In the NCB, trust and 
transparency are the basic means for 
stakeholders to determine whether all 
partners are meeting their commitments. 


The first accountability 
and performance 
report on the NCB is 
due this spring. 
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Social Programs 


There is a risk that 
savings are being 
realized before 
reinvestment 


programs are in place. 


Proposed outcome 
indicators may not 
reflect very accurately 


the impact of the NCB. 


The First Accountability and 
Performance Report 


6.77 Federal-provincial-territorial 
partners have said that the first National 
Child Benefit accountability and 
performance report “will outline the 
results of the NCB to date and detail the 
performance measures that will help us to 
remain publicly accountable for the 
success of the NCB.” That report, 
originally scheduled for September 1998, 
was due at this writing in the spring of 
1999. Our observations are based on the 
description of the report in the 
Governance and Accountability 
Framework released after the March 1998 
meeting of Ministers Responsible for 
Social Services. 


6.78 The partners stated that the goals 
for the report are to: 


e place the NCB in the context of child 
poverty; 


e establish some baseline data; 


e provide some statistics on program 
recipients and expenditures; and 


e provide some preliminary 
information on how the program will be 
monitored and assessed, what indicators 
will be used, and what studies will be 
undertaken. 


6.79 The report is the core 
accountability document for the NCB. We 
recognize that in the first report it will be 
too early to comment on whether the NCB 
has achieved its aims. However, it is not 
too early to make some preliminary 
judgments about mechanisms for ensuring 
the quality and relevance of information 
to be gathered, and to discuss the 
relevance and limitations of the chosen 
performance indicators. Nor is it too early 
for the partners to say what targets and 
benchmarks they will use to monitor the 
program’s success. 


6-20 


Data for outcome indicators need to be 
accurate and comparable 


6.80 We believe the report needs to 
contain assurances from the partners that 
estimates are based on a consistent and 
appropriate methodology, and that data on 
savings and on actual expenditures have 
been (or can be) verified. In other words, 
the report should indicate actual amounts 
of federal tax benefits, as well as actual 
expenditures by the other partners. This 
will enable readers to determine to what 
degree each has met its financial 
commitments. There is a risk, particularly 
in the first year, that savings will already 
have been realized while some 
reinvestments may not yet have been 
made. It would be appropriate for the 
federal government to report the actual 
amounts of the NCB supplement broken 
down by province and other relevant 
analytical categories, and to include the 
total in Volume I of the Public Accounts. 


6.81 In the accountability and 
performance reports, the federal 
government and its partners have, in our 
view, individually and collectively 
committed to ensuring that relevant and 
comparable data are available. An 
important question will be whether this is 
accomplished. 


6.82 The partners have announced that 
the first report would contain two kinds of 
measures: 


e program Statistics to indicate the 
inputs and outputs of the NCB — who has 
benefited, overall expenditures, and so 
forth; and 


e outcome indicators — to assess the 
impact of the NCB on low-income 
families with children. 


6.83 The ministers said: 


Outcome indicators could include: 


e Changes in the percentage of the 
total income of low-income families 
that result from employment: 
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By monitoring this outcome indicator, 
we would be able to measure the 
degree to which low-income families 
with children increase their 
attachment to the workforce over 
time. 


This indicator would measure changes in 
attachment to the work force. However, 
the reasons for change might not be clear. 
For example, more stringent criteria or 
lower social assistance payments could 
also produce a positive trend in the ratio 
of employment income to 
non-employment income. This would not 
necessarily mean that child poverty was 
declining. 


e Changes in the number of 
families with children on social 
assistance: 


This indicator would reflect the 
degree to which low-income families 
with children move off social 
assistance, as compared to being in 
the workforce and/or receiving 
income supplements. 


This is probably a good indicator of 
general changes in the labour market. A 
lower proportion of families on social 
assistance would indicate that, all other 
things being constant, working was a more 
attractive option than receiving social 
assistance. However, it is less useful as a 
measure of NCB success, because 
attributing success or failure to the NCB 
would assume that labour market 
conditions and requirements for social 
assistance were constant. This is a large 
assumption. 


e Changes in depth of poverty: 


The NCB is an important step being 
taken to reduce the depth of child 
poverty over time. The depth of 
poverty indicates the income gap for 
families with children measured 
against one or more lines of 
low-income measurement. 


This indicator would show the NCB’s 
relative impact on family income 
assuming that, overall, economic 
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conditions remained constant. However, 
the numbers would need to be broken 
down by gender, location, family size, 
income level, type of employment, and so 
forth in order to be useful. 


6.84 Experts and advocacy groups 
were asked for their comments on these 
proposed measures. In response to their 
suggestions, the report will likely contain 
revised measures of outcome, offer 
caveats about their accuracy and validity, 
and present additional relevant 
background information for assessing 
NCB success. 


What the indicators tell us 


6.85 The word “indicators” tells us 
that the proposed measures will indicate 
whether or not things are going in the 
right direction, but not necessarily why. In 
complex situations, indicators are a useful 
diagnostic tool but not an unambiguous 
measure.of program performance. Also 
needed are measures of intermediate 
outcomes (such as the extent to which a 
decrease in the poverty gap can be 
attributed to the NCB) and program 
nauenen, Some outcome 
indicators may be 
more useful as 


diagnostic tools than 


6.86 In the booklet entitled “Building 
a Better Future for Canada’s Children”, 
the federal-provincial-territorial 
governments promise that the 
accountability and performance report will 


contain performance measures for the as program 
program as a whole: performance 
Governments are now identifying measures. 


performance outcomes that will 
measure the National Child Benefit’s 
success in achieving its objectives. 
They are also evaluating how closely 
results can be linked to the 
investments made in the National 
Child Benefit, recognizing that this 
program will be only one of many 
factors that will affect the well-being 
of children and families. 


Performance outcomes will be 
incorporated into public reports that 
will describe the National Child 
Benefit’s success in meeting its 
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Social Programs 


The accomplishments 
so far reflect a spirit of 
trust and 
collaboration. 


It will be important to 
carry through on the 
commitment to 
evaluation. 


objectives. These reports will ensure 
that governments are fully 
accountable to the public. They will 
also allow provinces and territories to 
evaluate and share the results of their 
various National Child Benefit 
reinvestment initiatives. 


6.87 Notwithstanding this optimism 
about measuring performance outcomes, it 
will be a great challenge to attribute 
outcomes to the impact of the NCB. For 
example, if more families leave social 
assistance to go to work, how much of this 
outcome can we attribute to the National 
Child Benefit and how much to general 
improvements in the economic climate 
and labour market? Conversely, if fewer 
parents leave social assistance to enter the 
work force, is it because the NCB 1s 
poorly designed? In both cases, there are 
other factors at play. It is important to deal 
openly with this issue. The most obvious 
reason is the time it will take to determine 
the NCB’s long-term impact on children. 
It could take 20 years to see whether they 
have entered the work force, gone to 
university and so on. This makes it 
essential that the report use measures of 
intermediate outcomes, and that it explain 
the degree to which measured trends and 
outcomes are attributable to the NCB. 


Role of evaluation 


6.88 How, then, can Parliament and 
other interested parties find out what the 
program has accomplished? As we have 
noted, good measures of intermediate 
outcomes will provide useful information. 
As well, evaluation studies will eventually 
provide insight on program results that are 
not easily captured by single measures. In 
the NCB’s Governance and Accountability 
Framework, the partners have said they 
“support the importance of evaluative and 
analytical work to ensure the appropriate 
and reliable measurement of program 
outcomes [and] recognize the importance 
of ensuring that data provided for 
purposes of public reporting [are] open to 
public scrutiny.” The evaluation 


framework is still being developed. As a 
result, there are not yet concrete plans that 
include both organizational and financial 
commitments to do the work set out in the 
framework, including ensuring that 
information systems are in place to show 
whether the NCB and similar programs 
lead to better outcomes for Canada’s 
children. It is important that this 
commitment be kept. The accountability 
report should report on the status of this 
work. 


Accomplishments to Date 


Reaching agreement: Clear evidence of 
common goals and trust 


6.89 Achieving federal-provincial- 
territorial agreement on a major social 
program and implementing it in less than 
two years is a significant achievement. In 
our interviews and in the documentation 
we saw from working groups of officials 
and meetings of ministers and deputy 
ministers, we found evidence of people 
working hard toward common goals. 
Many participants used the word “trust” to 
describe their relationships with each 
other and said that this program could not 
have got off the ground without it. 


6.90 There was a good level of 
co-operation in all of these groups and 
committees. When ministers met, for 
example, they used common briefing 
notes agreed on by their officials for each 
agenda item. Comments of those involved 
suggest that these meetings met the 
criterion (discussed in Chapter 5 of this 
Report) of dealing with complex 
relationships with co-ordination, mutual 
trust and confidence. 


6.91 A significant aspect of reaching 
agreement was the decision to link the 
goal of reducing child poverty to the goal 
of increasing parents’ attachment to the 
work force. While this linkage makes 
performance measurement a greater 
challenge, it is clear that agreement on 
these policy goals has been a critical 
factor in the progress to date. 
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Implementation: Continuing 
commitment to outcome measures and 
accountability 


6.92 The participants have displayed a 
commitment to transparency through 
frequent public announcements of most 
results of ministerial meetings, and in 
continuing statements committing to 
accountability. While some issues remain, 
the early stages of the process clearly 
reflect a desire to display an appropriate 
degree of accountability for this program. 


Transparency and consultation: Public 
was informed at many key steps 


6.93 A potentially important aspect of 
a commitment to transparency is 
consultation. When the target group is 
children living in poverty, direct 
consultation on relatively complex matters 
related to reinvestment strategies and 
selection of outcome measures is very 
difficult. Proxies must do. To provide a 
forum for exchange on NCB issues, 
HRDC set up the NCB Reference Group 
with representatives from 10 groups of 
non-governmental organizations that are 
concerned about children in poverty. 
Although the consultations were not 
public, the recommendations from these 
groups were considered in the NCB 
Working Group’s deliberations. As well, 
some jurisdictions undertook public 
consultations about their reinvestment 
strategies. 


6.94 At several points in the process, 
public announcements were made. An 
Internet site was established that provided 
all the documents released after meetings 
of Ministers Responsible for Social 
Services (http://socialunion.gc.ca/ 
ncb_e.html). All this was in addition to 
special efforts by Revenue Canada to 
inform the public and respond to its 
concerns in order to ensure as smooth a 
transition as possible when the new 
benefit was implemented in July 1998. 
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Conclusion 


6.95 In observing the early stages of 
the National Child Benefit, we recognize 
the special effort participants are making 
to ensure accountability. There are 
potential gaps in the quality of financial 
and other information that will need to be 
addressed. Outcome indicators will not 
reflect the impact of NCB programs alone. 
It will therefore be important to have 
realistic expectations about what the joint 
accountability and performance reports 
can say, in the short term, about the 
impacts of the NCB. Nor is it clear to 
what degree the information supplied by 
all parties will be accurate and verifiable. 
This includes information from those who 
helped frame the agreement (provinces, 
territories and the federal government), as 
well as those responsible for designing 
and implementing only specific programs 
(Ontario municipalities and First Nations). 
It is not clear if data will be sufficiently 
comparable to allow for comparison of the 
outcomes of the different reinvestments 
made under the NCB. 


6.96 The challenge for all involved in 
this program is to be able to assure 
taxpayers that moneys are spent with due 
regard to economy and efficiency, and 
with appropriate means to measure and 
report on the effectiveness of not only 
individual parts of the program but also 
the program as a whole. This needs to be 
done in a way that respects the 
jurisdictional competence of the different 
parties involved. We believe that it is 
possible to do so and that many steps in 
the right direction have already been 
taken. Indeed, if the issues we have 
discussed are successfully addressed by all 
parties concerned (including audit 
offices), accountability for the 
expenditures and outcomes of the National 
Child Benefit will be just as rigorous and 
comprehensive as it would had only one 
jurisdiction been responsible. 


Social Programs 


Comparable data are 
needed to see what 
works and what does 
not work. 


We believe that 
accountability for this 
shared program can 
be achieved if the 
issues raised in this 
study are successfully 
addressed. 
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How to achieve accountability for the 
NCB: Five suggestions 


6.97 To parliamentary committees and 
others who may want to examine the early 
reports on NCB expenditures and 
outcomes, we offer five suggestions 
consistent with the general questions 
about collaborative arrangements 
discussed in Chapter 5 of this Report. 


e Ask how and when we will know if 
desired results have been achieved. In 
other words, have reasonable expectations 
for the accountability reports—at current 
levels of funding, the NCB cannot be 
expected on its own to fully eliminate 
child poverty or the “welfare wall”. Some 
impacts will not be visible in the first or 
even the second year, but means of 
collecting the data to show the impacts 
must be in place now. 


e Ask how reliable the information is. 
How good are the data? How can one 
know? Information could be unverified 
and still be reliable, but in what 
circumstances should only those supplying 
the information make this judgment? 
Where there is potential for conflict of 
interest (an interest in reporting mainly 
positive results, for example) ask not only 
whether there is a need to consider 
obtaining independent assurance but also 
whether doing so would be worth the cost. 


e Accept that the choice of outcome 
measures is partly political. It is 
reasonable to question whether a chosen 
measure is appropriate, but not whether 
final accountability for the choice should 
rest with elected officials. Consider asking 
under what circumstances it might be 
appropriate to delegate the responsibility 
to independent third parties for 
implementing or revising the measures 
and evaluating the results. 


e Encourage transparency and 
openness by treating the publication of 


negative trends in outcomes as 
opportunities for learning or improving 
the program, not for laying blame. 


e Accept that in an interjurisdictional 
situation like the NCB, accountability is 
shared. Seek answers from those best able 
to answer. Overall accountability can be 
achieved only through holding partners 
accountable for not only their own 
programs but also overall outcomes. 


Department’s comment: The Department 
appreciates the thoughtful analysis that 
the Auditor General provided in this case 
study on accountability for the National 
Child Benefit (NCB). The study raises 
some important issues surrounding the 
establishment of appropriate 
accountability arrangements and the 
credible measurement of results. The 
comments and suggestions of the Auditor 
General will be of considerable assistance 
to the Federal-Provincial-Territorial NCB 
Working Group as it prepares the NCB 
Progress Report and continues to build a 
NCB accountability regime. 


We can report that many of the issues 
raised by the Auditor General are being 
addressed by the federal government and 
its provincial and territorial partners. For 
example, outcome indicators and 
evaluation issues have been the subject of 
consultations with experts and advocacy 
groups and work will continue on these 
issues. In addition, the 
Federal-Provincial-Territorial NCB 
Working Group is currently preparing the 
first NCB Progress Report. All efforts are 
being made to ensure that the information 
contained in the Progress Report is timely, 
comparable, credible and complete. The 
remaining issues identified by the Auditor 
General will be raised by the federal 
government in its capacity as Co-Chair of 
the NCB Working Group. 
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Employability Assistance for People with Disabilities 


Introduction 


6.98 During 1997-98, provinces 
(except Quebec) and Human Resources 
Development Canada (HRDC), on behalf 
of the federal government, agreed on a 
framework for a new initiative: 
Employability Assistance for People with 
Disabilities (EAPD). Quebec did not 
endorse the framework, but in 1999 it did 
sign a bilateral agreement with the federal 
government to participate in EAPD. 
Implementation began in 1998-99, with 
plans to be fully operational by 2001-02. 
This is a $193 million shared-cost 
contribution arrangement (annual 
provincial costs are matched by the 
federal government), which is described 
more fully in the federal government’s 
Social Union Internet site (http://social 
union.gc.ca). As a financial arrangement, 
it is similar to other shared-cost programs, 
including its immediate predecessor, 
Vocational Rehabilitation of Disabled 
Persons. 


6.99 The EAPD initiative represents a 
small, targeted portion of total spending 
on persons with disabilities. Its objectives 
emphasize employability; that is, it 
supports programs aimed at helping 
people with disabilities achieve the 
capacity to enter and remain in the regular 
labour market, as distinct from supporting 
participation in sheltered workshops and 
related kinds of employment supports. 


6.100 This case study focusses on the 
accountability arrangements of EAPD and 
the elements to be included in future 
reports on its results, particularly the 
performance indicators that have been 
identified and the challenges implicit in 
designing and using these indicators. Our 
aim was to assess whether EAPD has 
some of the characteristics of a 
collaborative arrangement, to identify the 
factors that may delay or constrain the 
achievement of appropriate accountability 


arrangements and to make suggestions for 
improvement at an early stage in the 
process of implementing this initiative. 


6.101 As the population ages, the 
numbers of persons with disabilities will 
increase. The most recent Statistics 
Canada data (1991, based on the 
since-discontinued Health and Activity 
Limitation Survey) indicate that 

16 percent of Canadians (or approximately 
4.2 million people) report some level of 
disability. Fifty percent of those over 65 
report some level of disability. 


6.102 The last five years have seen 
major efforts to revise and improve 
services while reducing costs. The 1994 
government-wide Program Review, the 
Mainstream ’92 initiative and the 1996 
report of the Federal Task Force on 
Disability Issues (the Scott Task Force) 
have all had a significant impact. The 
Standing Committee on Human Resources 
Development and Status of Persons with 
Disabilities produced four reports between 
1990 and 1995, dealing with such matters 
as the legislative and regulatory 
framework, the continued 
“marginalization of persons with 
disabilities”, improvements to the tax 
system, and the integration of persons 
with disabilities from the concept of a 
“warehouse” toward an “open house”. 


6.103 The process of arriving at a new 
program arose from federal-provincial- 
territorial discussions on social policy 
renewal. Annual Premiers’ Conferences, 
First Ministers’ Meetings, meetings of 
Ministers Responsible for Social Services, 
and meetings of the Federal-Provincial- 
Territorial Council on Social Policy 
Renewal all made it a priority to improve 
income support for disabled persons 
through increased employment. Indeed, 
one measure of program success would be 
savings realized by income support 


Social Programs 


There are 4.2 million 
Canadians who report 
some level of 
disability. 


Agreement on the 
initiative emerged from 
a process of 
successful 
federal-provincial- 
territorial 
collaboration. 
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Social Programs 


A multilateral 
framework served as a 
guide for subsequent 
bilateral negotiations. 


6-26 


programs due to increased labour market 
participation by persons with disabilities. 
In addition to the clear benefits for 
persons with disabilities, their increased 
pay would mean increased government 
revenue and reduced income support 
expenditures over the long term. With this 
direction in mind, in February 1997 
officials began to review programming 
then funded under the Vocational 
Rehabilitation of Disabled Persons Act. 


6.104 The first order of business was to 
develop a mutually acceptable multilateral 
framework that would guide the 
negotiations of subsequent bilateral 
contribution agreements between the 
federal government and the provinces and 
territories. Three informal task teams 
worked in tandem to develop an 
accountability framework, proposals for 
changes to the funding allocation formula 
and a policy framework for EAPD. The 
chief policy thrusts are a clearer focus on 
employability and accountability, 
bolstered by commitments to measuring 
results and to ongoing evaluation of the 
program. HRDC held formal consultations 
with representatives of the community of 
people with disabilities in April and June 
1997. Provinces also incorporated the 
input from their own consultations into the 
work of the Federal-Provincial-Territorial 
Working Group. 


Observations 


Governance and Accountability 
Arrangements 


6.105 The multilateral framework and 
the bilateral contribution agreements have 
several of the elements discussed in 
Chapter 5 of this Report, on collaborative 
arrangements. The governments agreed to 
a co-ordinated, participative planning 
process and to evaluation of program 
results. The agreements also call for both 
a quantitative and a qualitative approach 
to assessing performance. Results 


indicators are stated, taking into account 
the characteristics of people with 
disabilities, the desire to achieve 
administrative efficiencies and the value 
of stakeholder participation. 


6.106 The partners agreed that the 
initiative would emphasize “annual 
accountability to consumers and the 
general public.” As we discuss more fully 
in the case study on the National Child 
Benefit (paragraph 6.56), accountability to 
the public need not diminish 
accountability to Parliament; the key is 
that information must be adequate so 
reporting is credible. 


Performance measures are varied: 
Some present challenges to implement 
and use 


6.107 The multilateral framework 
recognizes that the program is accountable 
to funders and to the public; results in the 
short, medium and long terms must be 
monitored and assessments of results 
should involve the client groups. The 
framework also recognizes that not all 
results are easily quantified, so it allows 
for qualitative assessments such as “best 
practices”. Use of both qualitative and 
quantitative measures is expected to 
produce more useful and balanced 
information than either type alone. 


6.108 The primary quantitative 
indicators of results focus on the “supply 
side” — increasing the number of 
employable persons with disabilities who 
are ready to enter the labour market. Other 
programs for persons with disabilities 
have dealt with the “demand side” of the 
equation — stimulating the demand for 
employees who are disabled by working 
with employers, business and community 
groups to change stereotypes, identify 
appropriate training and job opportunities 
and demonstrate how persons with 
disabilities can fit into the workplace. The 
challenge is to ascertain, when indicators 
show a positive or a negative trend, 
whether employment opportunities have 
been affected by external labour market 
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factors (such as a general rise or fall in the 
unemployment rate) rather than by EAPD. 


6.109 Our 1997 Report Chapter 22, 
Crown Corporations: Making Performance 
Measurement Work, describes a good 
performance indicator as one that is 
meaningful, reliable and practical. 
“Meaningful” means the indicator is 
understandable (clear, concrete/measur- 
able, in context), relevant (related to 
objectives, significant, useful, 
attributable) and comparable (over time 
and with other organizations or standards). 
“Reliable”, in this context, means it is an 
accurate representation (valid/free from 
bias) of what is being measured, is 
replicable or verifiable, is free from error 
or manipulation and balances other 
indicators. “Practical” means that it is 
feasible, both financially and temporally, 
to gather data. 


6.110 The quantitative indicators 
described in the multilateral framework 
were developed through an analysis of 
EAPD’s component programs, the input of 
the client community, and HRDC’s 
experience with results indicators in the 
Employment Insurance program. The 
measures tend to focus on activities and 
intermediate outcomes rather than on 
long-term results. The indicators agreed 
on in the multilateral framework show 
some of the challenges that arise when 
trying to measure results, as the following 
paragraphs discuss. 


6.111 Number of people employed, or 
sustained in employment in the event of 
vocational crises. This indicator actually 
contains two measures that have meaning 
and relevance, where it is feasible to 
gather the data. The bilateral agreements 
recognize this and split them. However, 
the task of defining the measures is not 
finished. For instance, the partners will 
need to define operationally what 
“sustained in employment” means. 


6.112 Number of people actively 


participating in or successfully 
completing an EAPD program and, if 
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unsuccessful, the reasons. The number of 
active participants is meaningful as a 
“throughput” measure, and 1s practical to 
collect. Determining why participants may 
not have succeeded could provide 
important information for lessons learned 
and program redesign. However, unless all 
jurisdictions have similar definitions of 


what it means to complete a program, and Defining good 
similar methods of data collection, the 

information will not necessarily be performance 
comparable. indicators is a 
6.113 Number of people not served, challenge 


on waiting lists or unable to access 
interventions. This indicator in the 
multilateral framework contains three 
measures, the first and last of which were 
omitted from all bilateral agreements. The 
“number of people on waiting lists” is a 
measure of unfulfilled demand for 
services, but different standards and 
practices for compiling waiting lists in 
different jurisdictions will cause data 
variations, making comparisons difficult. 
Length of time on the waiting list is also 
important. Several bilateral agreements 
therefore added a measure for “waiting 
periods”. 

6.114 Savings to income support 
programs as a result of increased 
earnings through employment. This 
potentially valuable indicator presents 
major challenges of practicality in 
obtaining accurate and reliable data. The 
federal-provincial accountability task 
team recognized this complexity and 
proposed that “all provinces should be 
engaged in a discussion around the 
methodological design required to capture 
this element”. The idea behind the 
measure is simple — a person whose 
employment income increases as a result 
of this program will need less income 
support or perhaps none, enabling 
government to save correspondingly. 
However, the question immediately arises 
as to whether, for how long, and in what 
proportion the disabled person’s increase 
in income is to be claimed as “savings” to 
income support programs. The answer will 
require good research into the causal 


Partners must all use 
the same definitions 
for terms like 
“successful” so that 
measures of 
performance are 
comparable among 
jurisdictions. 
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The implementation 
issues seem solvable. 


Roles and 
responsibilities for 
evaluation have yet to 
be specified. 


linkages among EAPD, employment 
earnings and savings to income support 
programs. Virtually all bilateral 
agreements state that joint evaluation of 
this indicator will be “subject to the 
feasibility of collecting the data and 
establishing the systems linkages 
required”. A further complication is that 
federal and provincial privacy laws must 
be respected. 


6.115 Number of people who have 
received support and have maintained 
employment or advanced in their jobs. 
This indicator was incorporated into only 
one bilateral agreement. The other 
indicators already capture most of the 
required information. Advancement in 
employment would be useful to know but 
actually obtaining the data would present 
significant challenges: the meaning of 
“advancement” would have to be defined 
in ways that would allow for a comparison 
of the results of different programs in 
various jurisdictions. 


6.116 In general, it is too early to assess 
how successfully the program has 
addressed the challenge of gathering and 
presenting data that are reliable and 
comparable across jurisdictions and that 
reflect the program’s performance. The 
partners are aware of the challenge. For 
example, they have agreed to modify in 
the bilateral agreements the primary 
indicators agreed to in the multilateral 
framework. They agreed that some of the 
indicators would be reported in annual 
reports; participants would use formal 
evaluation to measure the others and to 
assess the reasons for any lack of success. 
This is clearly a step in the right direction. 


Roles and responsibilities are stated in 
general terms 


6.117 The bilateral agreements set out 
the general roles and responsibilities of 
the parties; this general approach was 
helpful in reaching agreement. The most 
fundamental distinction is that while 
partners share responsibility for the 


overall framework and the contents of 
each bilateral agreement, the provinces 
deliver EAPD programs and the federal 
government pays half the cost. Many 
specific details, such as roles and 
responsibility for evaluation, have yet to 
be specified. Officials involved in the 
negotiations state that the complexity of 
the program and the tight time frame in 
which the bilateral agreements were 
negotiated tended to preclude covering 
these implementation issues in the 
agreements. As we note in the case study 
on the National Child Benefit 

(paragraph 6.41), negotiations typically 
concentrate on general principles in order 
to reach general agreement. The resultant 
flexibility and room to manoeuvre can, 
appropriately, leave many important 
details to be worked out later. However, 
implementation should not be an 
afterthought. Those responsible for 
program design, delivery and oversight 
must all pay close attention to the issues it 
raises. We saw no evidence that the 
partners have encountered implementation 
issues that are insoluble. 


It is too early to say whether reporting 
will be credible 


6.118 We are encouraged by the 
commitment to accountability made in the 
multilateral framework. We are also aware 
that informal federal-provincial task teams 
have been working since 1997 on 
performance indicators and on specifying 
the data needed. Work is now 
commencing on establishing evaluation 
parameters. Given the lead time and the 
planning that has already taken place, we 
expect that eventually there will be good 
information on what works and what does 
not work. 


6.119 Readers of accountability reports 
will want to ask the following kinds of 
questions. 


6.120 Are the data presented 
reasonably accurate and is the degree of 
any inaccuracy known and acceptable? 
One important aspect of reporting credibly 
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on program results is describing the 
reliability and validity of the information 
presented and the methods used to collect 
and verify the data, giving an indication of 
their accuracy, and noting any related 
factors that may have influenced the 
results. The technical and practical 
challenges of obtaining valid, reliable and 
consistent data are many, including the 
challenges of attributing results to the 
measures and tracking clients. The parties 
will need to continue working to resolve 
many of these issues if information on 
results is to be credible, timely, 
meaningful and useful. 


There is no requirement for independent 
verification of other performance 
information. We believe that the federal 
government should seek some assurance 
about the accuracy of the systems that 
provide the information. 


6.124 The bilateral agreements also 
require the provinces each to produce 
annual reports and multi-year program and 
expenditure plans. These annual reports 
are to be published. However, it is not 
clear that they will contain comparative or 
analytical cross-provincial data. We 
believe this type of data would contribute 
to accountability for the federal 
expenditure, by providing information to 


Accountability is 
better served if data 


6.121 Is there knowledge of what data are comparable across 


gaps and problems exist and are there 
plans in place to fix them? There may 
be, for example, differences among the 
parties in their standards of program 
completion, measurements of waiting 
lists, and calculations of savings to 
income support programs. We are aware 
that this is a concern for officials, but we 
do not know precisely what plans they 
have to deal with these issues. 


6.122 Is there reasonable provision 
for review, audit and program 
evaluation? The parties have agreed to 
review the arrangements under the EAPD 
agreements after the end of the third year 
(2000-01). We expect that all parties will 
routinely review and adjust their 
programs, in order to improve services 
and capitalize on best practices and 
lessons learned from pilot projects. The 
multilateral framework commits the 
partners to reasonable review efforts 
(which normally would result in 
adjustments to improve efficiency and 
effectiveness, and in more transparency). 
The key will be how these review 
provisions in the framework are 
implemented. 


6.123 The bilateral agreements require 
the provinces to produce audited financial 
statements of program expenditures. 
These statements are sent to HRDC for 
approval and reimbursement of expenses. 


identify best practices and to make valid 
comparisons of provincial program 
performance. We recognize that 
comparisons can be misinterpreted and 
oversimplified. It will be important to 
make the appropriate qualifications in the 
overall annual report that we recommend 
HRDC prepare (see paragraph 6.129). 


jurisdictions. 


6.125 The agreements also require 
provinces to conduct program evaluations 
of longer-term impacts (though no due 
dates are specified) and to measure 
progress using the identified performance 
indicators. These evaluations may serve to 
confirm the provinces’ performance 
information. HRDC, as one of the parties, 
agrees to undertake ongoing evaluations 
with its partners and to make the 
evaluation results public. The partners are 
aware of the challenges to fulfilling the 
evaluation requirement. A group of 
provincial and HRDC regional officials is 
working out the details of how, when and 
by whom the various aspects of evaluation 
will be undertaken. HRDC’s Evaluation 
and Data Development group is actively 
involved in defining approaches to the 
EAPD evaluation and has developed a 
discussion draft Proposed Framework for 
the Evaluation of EAPD. Two 
federal-provincial accountability 
sub-working groups were created to 
develop options for the evaluation and to 
establish a common set of definitions for 
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In the final analysis, 
success will be judged 
by results, not by what 
is in the agreements. 


the results indicators. This presumably 
will involve identification of common 
issues, supporting data, time frames, roles 
and responsibilities, as well as an 
approach to national issues and a national 
synthesis of results. So far, the majority of 
provinces have agreed to proceed with an 
assessment of the feasibility of evaluations 
that would examine the design and 
delivery features of the EAPD 
interventions existing across Canada. This 
work may also determine if it will be 
possible, given the data being collected, to 
carry out a longitudinal survey of EAPD 
participants and to follow their outcomes. 


Conclusion and 
Recommendation 


Accountability, performance measures 
and evaluation remain at issue 


6.126 The process by which the two 
levels of government arrived at the 
bilateral agreements for EAPD contains 
many worthwhile elements that illustrate 
the precepts outlined in Chapter 5 of this 
Report, Collaborative Arrangements: 
Issues for the Federal Government. The 
degree of co-operation and collaboration 
involved in reaching these agreements 
deserves praise. However, in the final 
analysis, the success of EAPD will be 
judged not by what is in the agreements 
but by the results they generate. 
Nevertheless, adherence to the agreements 
may contribute to program success. Have 
the evaluations actually been undertaken? 
What do they tell us? Have gaps in data 
been successfully addressed? With 
implementation still under way, it is too 
early to have definitive answers to such 
questions but it is not too early to ask 
whether plans and budgets indicate a firm 
commitment to realize these aims. At this 
early juncture, we have two observations 
and a recommendation for improvement. 


6.127 There are no plans to publish 
any overall annual report. The first 


provincial annual reports will be produced 
for the year 1999-2000. However, no 
deadline has been set for tabling or 
submitting these reports and there is no 
provision for an overall annual report. 


6.128 There are no specific goals or 
targets over any specified time frame. 
HRDC officials cite the lack of baseline 
data and the complexity of the services 
delivered by provinces and third parties as 
reasons for the lack of defined 
performance expectations or established 
targets. They also consider it more 
important to meet the needs of individuals 
than to set targets, though some provinces 
have made some projections. Stakeholders 
and officials are also concerned that 
setting such targets would encourage 
“creaming” — that is, selecting first the 
clients with the greatest chance of success 
so as to yield early positive results. This 
would discriminate against persons who 
need longer and/or more interventions to 
achieve success. While this means that 
overly simplistic targets are inappropriate, 
we believe that well-defined goals can, 
and still should, be established for each 
EAPD program. 


6.129 To add to the annual reports by 
the partners, Human Resources 
Development Canada should, in 
collaboration with its partners, prepare 
a brief overall annual report on 
Employability Assistance for People 
with Disabilities, comparing partners’ 
activities, expenditures, program 
outputs and performance. 


6.130 The report should include: 


e performance targets both for the 
program overall and for the individual 
provincial programs; 


e adescription of limitations in the 
accuracy and the comparability of data; 


e an indication of which data have 
been audited; 


e information on program 
adjustments and lessons learned from 
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arrangements; 


e« performance measurements; and 
e evaluation studies. 


6.131 This annual report should be 
made public not later than six to nine 
months after the end of the fiscal year. 


Department’s response: The Department 
appreciates the contribution of this case 
study to our work with provinces in 
implementing Employability Assistance 
for People with Disabilities (EAPD). The 
Auditor General has outlined many of the 
key challenges we face in meeting the 
commitment to accountability and results 
indicators agreed to by federal and 
provincial-territorial Ministers 
Responsible for Social Services. The case 
study provides a useful analysis that will 
inform our ongoing work with provinces. 


Human Resources Development Canada — Accountability for Shared 
Social Programs 


The Department agrees in principle with 
the Auditor General's recommendation to 
produce an overall annual report on 
EAPD. Work is under way, in 
collaboration with provinces, to ensure 
consistency in reporting on EAPD-funded 
programs. The Department will consider 
the suggestions and comments of the 
Auditor General in the development of 
annual report. In preparing the reports 
and in measuring and comparing results 
and outcomes, it will be important to note 
that EAPD-funded programs constitute 
only part of overall programming for 
people with disabilities and should be 
considered in the unique context of each 
province. 
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% About the Study 


Objective and Scope 


Our aim was to help parliamentarians and others to arrive at a reasonable set of expectations for the 
accountability arrangements and measures that are important features of shared social programs in which 
Human Resources Development Canada (HRDC) is responsible for federal policy co-ordination and 
leadership. Our scope included two case studies of programs in the “social union” — the National Child 
Benefit (NCB) and Employability Assistance for People with Disabilities (EAPD).We sought answers to the 
following questions: : 


e How can credible reporting on these two programs best be achieved? 
e What are realistic expectations for measuring the program outcomes? 
e Are there information gaps that will need to be filled? 


In each case study our aim was to help develop: 
e anunderstanding of potential accountability gaps and the context and factors that may delay or constrain 
improvement in accountability and results measurement; and 


e an appropriate response to public accountability and performance reports, reports on social outcomes, and 
jointly designed performance management systems of these programs. 


Criteria 


Because this was a study of programs at an early stage of development, we did not have formal criteria 
against which to audit. We did nevertheless have expectations that could be used in a future audit. They 
derive from our 1997 Report Chapter 11, Moving toward Managing for Results. In particular: 


e indicators and measures should be clear and concrete; and 
e information should be understandable, balanced, attributable, and reliable. 


As well, for the National Child Benefit we expected that in the negotiations and in the joint accountability 
and performance report, the information ultimately would meet the following criteria: 


e Data on expenditures and for outcome indicators and evaluations are credible and comparable so that 
overall outcomes of the NCB can be measured and evaluated. The accountability report informs readers 
whether financial information has been audited, and whether arrangements are in place to ensure that due 
care has been taken with respect to the information presented, including data used for performance 
reporting. 


e Reinvestments are within the parameters of the NCB goals and objectives, and represent new programs or 
enhancements to existing ones. 


e There is knowledge of the data gaps and problems that exist and there are plans in place to fix them. 


e There are means to monitor and inform all participants and stakeholders about individual reinvestment 
expenditures, outputs and outcomes, as well as the outcomes of the entire effort. 
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Approach 


We examined relevant departmental documentation and conducted interviews with officials for each of the 
chosen programs, concentrating on the design and implementation of governance and accountability 
frameworks. We also reviewed literature on performance and outcome measurement in Canada and 
elsewhere, as well as transcripts of relevant hearings conducted by the Standing Committee on Public 
Accounts and the Standing Committee on Human Resources Development and Status of Persons with 
Disabilities. We also drew on the findings from our work on performance management and measuring results 
(1997 Chapter 11) and, from our work in this Report on collaborative arrangements (1999 Chapter 5). 


Study Team 


Assistant Auditor General: David Rattray 
Principal: Louis Lalonde 
Director: Lewis Auerbach 


Allison Fader 
Martin Dompierre 


Ali Jakubec 


For further information, please contact Lewis Auerbach. 
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Appendix A 
Auditor General’s Related Work 


Human Resources Development Canada (HRDC) 


The case studies are an extension of previous audits that have concentrated on the challenges for HRDC of achieving 
results and demonstrating accountability in very complex environments. This includes our 1997 Report Chapter 17, 

A Critical Transition toward Results-Based Management; and more recently our 1998 Chapter 16, Management of the 
Social Insurance Number, where responsibilities and impacts range widely between the federal and provincial 
governments and the private sector. 


Performance reporting 


In our 1997 Chapter 11, Moving toward Managing for Results, we said it is important to go beyond merely measuring 
inputs and outputs. 


Meeting the expectations of Canadians for programs that work and finding more cost-effective means of 
delivering them requires managing for results. When managing for results, ministers, senior officials, managers, 
staff, central agencies and Parliament make decisions based on what a program is achieving for Canadians — the 
results that citizens value — and at what cost. In particular, there is a focus on outcomes, that is, the benefits 
realized. Putting this approach into practice means agreeing on expected outcomes, measuring progress toward 
them, using the information and reporting results. 


Quality of provincial information 


Our interest in the quality of provincial information is not new. We have been commenting on aspects of 
interprovincial agreements for many years. For example, 10 years ago we observed that federal officials responsible 
for the Canada Assistance Plan did not know how much reliance they could place on provincially supplied 
information. Nor was this problem immediately corrected. In our 1991 Chapter 3 follow-up, we stated that the issue 
was being addressed but that information to Parliament was still not adequate. 


More recently, in our 1996 Chapter 26, Canada Infrastructure Works Program — Lessons Learned, we observed that 
provincially supplied information had not been verified: 


In the majority of files we examined, federal officials endorsed provincial assessments without requiring direct 
supporting information, or at least ensuring that appropriate information had been provided to, and analyzed by, 
the province concerned. 


The Commissioner of the Environment and Sustainable Development is also concerned about achieving accountability 
when several jurisdictions are involved. He said in his 1998 Chapter 3, Responding to Climate Change, that while 
many jurisdictions were involved in climate change issues, ultimately the federal government is accountable for the 
overall structure: 


We believe that undefined and diffused accountability erodes the authority and responsibility of the federal, 
provincial and territorial governments and their officials and makes it difficult for anyone to measure their 
respective performance. It also means that accountability for Canada’s progress toward its stabilization goal is 
obscured. Nevertheless, in our opinion the federal government remains accountable to Parliament for establishing 
an effective management structure to respond to climate change. 
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Appendix B 


Historical Context — National Child Benefit 


Beginning with a children’s income tax exemption in 1918 and universal family allowances in World War II, and 
continuing throughout the 1960s and ‘70s, many federal programs such as child care tax credits were specifically 
designed to improve child welfare. Others such as Medicare and Unemployment Insurance had outcomes that, by 
improving the situation of parents, clearly benefited their children. 


Early social programs of the ‘40s, ‘50s, and “60s such as Unemployment Insurance, Old Age Assistance and Family 
Allowance were direct federal expenditures administered entirely by the federal government. Each such program 
required an Act of Parliament and, unless funded separately like Unemployment Insurance, required annual 
appropriations as part of the federal Budget. The Office of the Auditor General can, and did, audit the expenditures as 
well as the administration of such programs. As well, parliamentarians had opportunities through the Budget debates to 
question the expenditures and the administration of these programs. 


The 1978 revisions to the Family Allowance Act increased child allowances but made them taxable. Making child 
allowances taxable had the effect of targeting the benefit more toward lower and middle income families. From an 
accountability perspective, the relationship of Parliament and the nature of program delivery remained the same — the 
program was still an expenditure in the Budget cycle. The Child Tax Credit introduced in 1978 to further supplement 
the Family Allowance for families of modest means was an even more significant change. Because tax credits form a 
part of the income tax system, the credit did not require annual review by Parliament; the form of financial assistance 
was not direct payment but rather forgone revenue, that is, a tax expenditure. 


In 1992, family allowances were eliminated completely and replaced by the Canada Child Tax Benefit. One 
consequence for accountability (as we observed in our 1994 Report Chapter 6, Study of Key Federal Social Programs) 
was that “because the new Child Tax Benefit is now delivered through the tax system, it is not included in budgetary 
spending. Thus, it becomes difficult to make comparisons or analyze trends in program spending in this area.” In our 
1998 Chapter 8, Effectiveness Measurement and Reporting in the Department of Finance (which is responsible for tax 
policy and evaluations of taxes and tax expenditures), we noted that effectiveness measuring and reporting of the child 
tax benefit were “insufficient”. 


Whatever the shortcomings of past child support programs, at least the role and responsibility of the federal 
government was clear. In the case of the 1992 changes, for example, a budget was presented and debated, and 
Parliament enacted the necessary changes to the Income Tax Act and replaced the Family Allowance Act. 


Current situation 


With the National Child Benefit (NCB), accountability is less well defined. Responsibilities are negotiated and shared. 
Moreover, there is no single piece of legislation that mandates the program. While this means that the potential exists 
for gaps in accountability, federal and provincial partners are aware of the potential problems and are working to 
resolve them. The co-operation we received from officials and the documentation we reviewed confirm that the 
partners in the NCB also wish to see good accountability arrangements in place for this program. However, it is too 
early to judge that they have succeeded. 


Policy context: Research results demonstrate importance of early childhood 


Over the last 10 years, research has convincingly demonstrated that very early childhood experiences can have lasting 
impacts, both positive and negative. Moreover, poverty is associated with many of the negative indicators, such as 
poor health or low birth weights. The government summarized this research in 1997: 


Most Canadian children are doing well and are physically, emotionally and socially healthy. Unfortunately, some 
children are not doing as well. Many experience emotional, behavioural or learning problems that affect school 
performance and personal development. Others suffer from physical problems such as disease, disability and 
injury. 
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Such difficulties can and do affect children being raised in all family situations—in two-parent as well as 
single-parent families, in every neighbourhood and community throughout the country. But decades of research in 
Canada and abroad have shown that low family income is strongly linked with these problems. Compared to 
children from middle- and upper-income families, poor children run a greater risk of inadequate nutrition, mental 
and physical health problems, poor school achievement and difficulties with the law. 


Living on a low income can exert a heavy toll not only on children and their parents but also on the economy and 
society. Although many low-income children grow up and out of poverty, its harmful effects in childhood can 
often linger long into their adult years. The costs include increased expenditures for welfare, employment 
insurance, social services, health care, the courts and the penal system; lost tax revenues; wasted productivity; and 
missed opportunities for both individuals and society. 


The 1997 federal Budget 


The 1997 federal Budget announced the goals and major premises for a National Child Benefit (http://www. fin.gc.ca/ 
budget97/childe/childe.html). These goals linked a commitment to reduce poverty, with the concern that welfare 
systems create disincentives for parents to work. The Budget papers explained this as follows: 


e Right now, the combined effect of federal and provincial programs is to reduce the child benefits of parents 
who leave welfare to enter the workforce. Parents should not be put in the position of penalizing their children in 
order to take a job. 


e [The benefit would be directed to working families, who would see their incomes rise. Although the 
intention was to reduce social assistance by the amount of the benefit] ...the overall income of families on social 
assistance will be protected. 


e In addition to being a win for Canadian children, the National Child Benefit system is a step forward in 
Canadian federalism, with the federal, provincial and territorial governments seizing on a good idea, setting 
common objectives and working as partners to secure better lives for our children. 


e To accomplish this, an enriched federal benefit along the lines set out above will require provinces and 
territories to make offsetting reinvestments of provincial funds to assist children in low-income working families. 
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Appendix C 


A Comparison — National Child Benefit and Employability Assistance for 
People with Disabilities 


In addition to the characteristics outlined in Exhibit C.1, the National Child Benefit (NCB) and Employability 
Assistance for People with Disabilities (EAPD) each have the following similarities and differences: 


Similarities 


e No specific legislative act governs either initiative; the parameters are an outcome of federal-provincial 
agreement. 


e First Ministers agreed on the program goals. 


e The program is not aimed at all members of the target group, but is primarily focussed on helping those who wish 
to enter or remain in the work force. 


e Program design responsibilities are delegated from First Ministers to Ministers Responsible for Social Services to 
their officials. 


e Provinces are responsible for program delivery (although many have arranged for Revenue Canada to deliver 
provincial and territorial tax benefits on their behalf). 


e The federal government’s role is to provide a substantial amount of the funding, to help establish an overall 
accountability framework, and to work with provinces to ensure that the program meets the basic overall objectives 
agreed to by the partners. 


Differences 


The differences reflect the fact that EAPD is more like a traditional federal contribution arrangement whereas the NCB 
is anew kind of arrangement. 


e In EAPD, the federal government’s financial contribution goes to provinces whose programs meet the mutually 
agreed criteria. In the National Child Benefit, federal funds go directly to individuals. 


e A formal, written multilateral framework for EAPD forms the basis for comprehensive bilateral agreements with 
provinces. For the NCB, the only bilateral agreements are operational agreements between Revenue Canada and 
provinces and territories for the transfer of information on recipients of NCB supplements, and administrative 
agreements to deliver certain tax benefit programs. NCB agreements on matters related to goals, accountability and 
governance are the public announcements and brochures that all participants approved. 


e For EAPD, each province has agreed to deliver its own annual report. HRDC is responsible for undertaking 
overall evaluations and assisting with individual evaluations, in collaboration with provinces. Federal and provincial 
officials are already meeting to establish the parameters for this work. There is no commitment yet to a periodic 
accountability report for the overall program like the joint annual report on the NCB. 


Report of the Auditor General of Canada — April 1999 6-37 


Human Resources Development Canada — Accountability for Shared 


Social Programs 


Exhibit C.1 


Characteristics of the National Child Benefit and 
Employability Assistance for People with Disabilities 


National Child Benefit 


Employability Assistance for People 
with Disabilities 


Current Status 


Implemented on | July 1998. 


Provincial reinvestment plans announced; some have yet to be 
designed and implemented. 


Quebec subscribes to principles, has relevant programs, but does not 
participate formally. 


Outcome measures are in final stages of design. 


Program started 1 April 1998. 


Bilateral agreements between federal 
government and provinces are now signed. 


Results indicators agreed to in multilateral 
framework. 


early 1999. Annual publication of the Report is mandatory under the 
Governance and Accountability Framework. Evaluations are 
envisioned, but no agreement yet on approach and funding. 


Ministers Responsible for Social Services approved a report outline 
in October 1998. 


All jurisdictions (except Quebec) will report on a designated set of 
indicators. 


Program Federal tax benefit for low-income families with children. Provincial delivery; federal government 
mae og Provincial programs, services, in-kind benefits for low-income oa eS eat CACO 
ae working families funded by “reinvestments” of provincial savings on 
social assistance to families with children. 
Accountability | Governance and Accountability Framework released in March 1998. Multilateral framework provides for results 
‘ 3 : : oe : al repo vinces 
Bk Emphasizes public reporting and public accountability rather than ERS Ca annualweports ey as 
se and involvement of persons with 
government-to-government accountability. er ee 
disabilities. 
Performance Outcome measures and performance indicators are still being Primary indicators include: 
Measures and designed. Consultations were undertaken to determine if there is : ; 
eS sie e number of people employed or sustained 
Results broad support for the proposed performance indicators. ; : 
; in employment; 
Indicators Mhe dics : Hed ae ; i oceil 
€ indicators are intended to measure: e number of people actively participating 
e reduction in depth of poverty; in or successfully completing programs; 
e increased parental attachment to labour force; e number of people on waiting lists; and 
e attributable improvements in child welfare; and e savings to income support programs as a 
; : en result of increased earnings through 
e reductions in overlap and duplication. 
employment. 
Evaluation First Accountability and Performance Report scheduled for release in Agreements require evaluation. Partners 


have agreed to assess types of data being 
collected, costs and benefits of evaluation, 
options for how, when, and by whom 
evaluations will be performed, and how they 
will be funded. 


Annual Cost 


Federal contribution for NCB supplement: $850 million (1998-99), 
increasing to $1.7 billion (2000-01). 


Provincial-territorial reinvestment commitment for June 1998—July 
1999: $359 million (Quebec excluded). 


First Nations commitment: $27.8 million. 


Reinvestments will increase as the NCB supplement increases. 


$193 million plus matching amounts from 
provinces. 
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formats you wish to receive. Mail the form to: 


Office of the Auditor General of Canada 
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Chapter 7 


The Atlantic Groundfish Strategy 


Contributions and Grants 


The audit work reported in this chapter was conducted in accordance with the legislative 
mandate, policies and practices of the Office of the Auditor General. These policies and 
practices embrace the standards recommended by the Public Sector Accounting and 
Auditing Board (PSAAB) of the Canadian Institute of Chartered Accountants. The 
numbered paragraphs in bold face represent recommendations. 
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The Atlantic Groundfish Strategy 


Contributions and Grants 


Main Points 


18 We have little assurance that all contributions under The Atlantic Groundfish Strategy (TAGS) were used 
for their intended purposes. These were part of TAGS active labour adjustment measures managed by Human 
Resources Development Canada (HRDC). 


eke Most of the deficiencies noted in the audit relate to a lack of diligence in assessing project proposals and 
signing contribution agreements, as well as lack of monitoring by the Department. 


Tod Many of the files contained no project proposals; in others, proposals were not sufficiently developed to 
allow proper assessment. Some agreements lacked complete information, included ineligible costs, or did not 
correspond to the measure under which the project was funded. There was little evidence of on-site monitoring 
visits to examine expense records. 


Background and other observations 


7.4 The Atlantic Groundfish Strategy, in effect from 16 May 1994 to 29 August 1998, comprised measures to 
assist those affected by the groundfish moratorium. About $150 million of the $1.9 billion allocated to TAGS was 

spent on active labour adjustment measures such as training, mobility assistance, wage subsidies and employment 

bonuses.. 


Tam Those expenditures were not covered in our audit of The Atlantic Groundfish Strategy, reported in 
Chapter 16 of our October 1997 Report. Following the publication of that Report, the House of Commons 
Standing Committee on Fisheries and Oceans requested that we audit the expenditures. 


7.6 Our Office and the Internal Audit Bureau (IAB) of Human Resources Development Canada agreed that 
IAB would audit the grants and contributions made under TAGS. We closely monitored and reviewed that audit so 
we could rely on the findings for this chapter. 


a TAGS had increased the caseload of officers by some 40,000 participants over a short period of time. 
This created pressure to identify, approve and contract for an unprecedented number of projects. 


7.8 Several contributions went to projects that were not clearly related to TAGS. More than half of the 
agreements were signed after projects had begun. Expenditures were reimbursed without supporting 
documentation for the claims. For more than half of the projects reviewed, there was no evidence that the files had 
been closed, although most of those projects had been completed more than two years earlier. 


Overall, Human Resources Development Canada agrees with the findings of the audit and recognizes that a 
number of important points and concerns are raised in the chapter. Actions undertaken to address the issues 
include the development of new policies and procedures, training for managers and staff and, in new 
initiatives, securing resources to ensure sufficient monitoring. 
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Introduction 


2 The Atlantic Groundfish Strategy 
(TAGS) came into effect on 16 May 1994 
(terminated on 29 August 1998), with the 
objective of restructuring the fishing 
industry in Atlantic Canada to make it 
economically viable and environmentally 
sustainable. This strategy comprised 
measures to assist individuals and 
communities affected by the groundfish 
moratorium. It applied to fishing areas in 
five provinces: Newfoundland, Nova 
Scotia, New Brunswick, Prince Edward 
Island and Quebec. 


7.10 Human Resources Development 
Canada (HRDC) was responsible for the 
labour adjustment component of TAGS, a 
series of active measures aimed at 
adjusting affected workers out of the 
fishery and enhancing the profession of 
those fishers who remained active in the 
fishing industry. Exhibit 7.1 outlines these 
measures and indicates the amount of 
expenditures spent on each of them. 


7.1 A key objective of the 
government was to ensure that the 
communities affected by the groundfish 
moratorium had sufficient funds to 
maintain essential economic and social 
activities. To that end, Human Resources 
Development Canada gave priority to 
income support for affected workers. To 
be eligible for income support, fishers and 
plant workers had to agree to participate 
in the active labour adjustment measures. 


7.12 As TAGS evolved and 
participation proved higher than had been 
forecast, funds designated for the active 
labour adjustment measures had to be 
transferred twice to income support (in 
January 1995 and July 1996). As a result, 
the requirement to participate in these 
measures was gradually abandoned. The 
last projects under TAGS were approved 
more than two years ago. 


gi We TAGS increased the caseload of 
officers by some 40,000 participants over 
a short period of time. This created 
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pressure to identify, approve and contract 
for an unprecedented number of projects. 


7.14 Moreover, given the lack of time 
available to plan and design the labour 
adjustment measures, HRDC developed 
some new options but mostly used those 
existing under another of its programs at 
the time, the Canadian Jobs Strategy, with 
little adaptation to reflect TAGS 
objectives. Furthermore, policies were 
introduced and changed as TAGS evolved, 
and so local managers had to be flexible 
and adapt accordingly. While 
implementing TAGS, the Department was 
also modifying its programs and reducing 
its work force. 


TAS Aspects of the labour adjustment 
component were discussed in our October 


Exhibit 7.1 


Active Labour Adjustment Measures 


May 1994 to September 1998 


Expenditures | 

Measures ($ millions) | 
{ = 

Contributions 
Mobility Assistance 22 
Employment Assistance 15.7 
Delivery Assistance 8.4 
Self-Employment Assistance 0.4 
Job Development 0.2 
Training: 90.4 
¢ Canada Employment Centre Purchases 
e Project-based Training 
¢ Workplace-based Training 
¢ (Co-ordinating Groups 
¢ “Green Projects” — Training 
Job Opportunities 4.4 
Community Opportunities Pool 13 
“Green” Projects — Regular 20.3 
Portable Wage Subsidies 2.6 

Grants 

University Tuition 1.0 
Employment Bonus 1.9 
Total | 148.8 


Source: Internal Audit Bureau, Human Resources Development Canada 
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1997 Report Chapter 16 on TAGS. 
Following the tabling of our Report, the 
House of Commons Standing Committee 
on Fisheries and Oceans passed a motion 
requesting that our Office audit the 
expenditures made under the grants and 
contributions administered by HRDC for 
the active labour adjustment measures. A 
copy of the Committee’s request is 
presented as an Appendix to this chapter 
(page 7-14). These expenditures represent 
about $150 million of the $1.9 billion 
allocated to TAGS. 


Focus of the audit 


7.16 Our objective was to respond to 
the request of the Standing Committee on 
Fisheries and Oceans and to provide 
assurance that projects funded under the 
active labour adjustment measures had 
met TAGS criteria; that they had been 
properly managed, controlled and 
monitored; and that the grants and 
contributions had been used for their 
intended purposes. Exhibit 7.2 provides a 
brief definition of grants and 
contributions. 


Exhibit 7.2 


Grants and Contributions — Definition 


Transfer Payments 


Grants and contributions are part of the category of expenditures known as 
transfer payments. These are transfers of money from the federal government 
to individuals and to organizations of various types, including businesses and 
other levels of government. 


Grants 


Grants are unconditional transfer payments for which eligibility and 
entitlement may be verified. If an individual or organization is eligible for a 
grant, the appropriate payment can be made without requiring the recipient to 
meet any future conditions. 


Contributions 


The payment of a contribution is subject to performance conditions that are 
specified in a contribution agreement. The recipient must continue to show 
that these conditions are being met in order to be reimbursed for specific costs 
over the life of the agreement. The government can also audit the recipient’s 
use of the contribution, whereas this is usually not a requirement for a grant. 
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Source: Policy on Transfer Payments, Treasury Board of Canada Secretariat 


Ths \7/ The Internal Audit Bureau (IAB) 
of Human Resources Development 
Canada had already planned an audit of 
grants and contributions provided by the 
Department. We agreed with IAB that it 
would conduct the audit, and that we 
would monitor and review its work to 
assess its relevance to our objectives and 
to determine whether the supporting 
evidence was appropriate for our 
purposes. 


7.18 Further details on our audit can 
be found at the end of the chapter, in 
About the Audit. 


Observations 


7.19 From our review of I[AB’s audit, 
we determined that the work was done 
well and that it followed professional 
standards. The audit results are presented 
in a report entitled “Audit of TAGS Grants 
and Contributions”. At this writing, the 
final report had been submitted to HRDC 
management for approval. 


7.20 The IAB audit team reviewed 
193 project files, including 

167 contribution agreements and 

26 grants. It also met, where possible, 
with sponsors and with employees at 
Human Resource Centres of Canada. Four 
general criteria were used: 


e The contracting process should 
comply with Treasury Board and HRDC 
standards for grants and contribution 
agreements. 


e Projects should be managed 
effectively and be monitored operationally 
and financially. 


e Sponsors should comply with the 
terms and conditions of their agreements. 


e Grants and contributions should be 
paid only to entitled individuals or 
organizations, and contributions should be 
used solely for their intended purposes. 


72M The audit identified important 
issues regarding the management of the 
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TAGS grants and contributions in this 
sample. The overall impact of the active 
measures on the fishers and plant workers 
of the affected regions was not considered 
in this audit. This has been addressed 
through program evaluation studies on 
TAGS. Exhibit 7.3 and the following 
observations present the main results of 
the IAB audit. 


Weak Management, Control and 
Monitoring of TAGS Contribution 
Agreements 


Proposals were not fully analyzed 


7.22 One third of the files examined 
contained no proposal to support the 
project. Of the 111 files that did contain a 
proposal, 33 percent did not meet the 
criteria for the labour adjustment measure 
under which they had been approved. In 
most files, it was impossible to determine 
how the selection criteria had been 
applied. As a rule, there was nothing in 
the file to indicate why a project had been 
recommended or selected. 


12s Few files contained enough 
information for the audit to verify whether 
the project sponsor had been entitled to 
the contribution under the measure for 
which the proposal had been submitted. 
Moreover, there was no indication that 
sponsors had been checked to see if they 
had any outstanding debt to the Crown, or 
were already receiving funds for similar 
activities. 


Many projects were not clearly linked 
to TAGS 


7.24 Many agreements did not state 
clear objectives for the project, and did 
not identify results that were measurable. 
In many cases, there was no clear 
indication that the project targeted TAGS 
participants, and several proposals did not 
state objectives related to TAGS. 


12> Certain projects, particularly 
where courses were purchased, included 
TAGS and non-TAGS participants but 
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expenditures for both groups were funded 
through TAGS alone, instead of under the 
proper funding authorities in proportion to 
TAGS and non-TAGS participants. 


Contribution agreements were 
undermanaged 


7.26 In general, pro forma agreements 
were used that had been developed for the 


Exhibit 7.3 


Audit of TAGS Grants and Contributions 


Sample Results 


Non-compliance 

Contributions (percentage of files) | 
Proposals 
Proposal on file 34 
Linkages with TAGS 28 
Compliance with eligibility criteria 33 
Inclusion of estimates 42 
Verification of debts owing to HRDC 80 
Verification of other contracts with federal agencies 84 
Supporting documentation for selection 83 
Agreements 
Signature prior to effective date Se, 
Clear project objectives 26 
Participant eligibility criteria clearly defined 43 
Establishment of progress measurements 66 
Amendments explained 43 
Monitoring of agreements 
Review by supervisor 47 
On-site monitoring of financial activities 76 
e Remedial action taken 19 
On-site monitoring of operational activities 74 
e Remedial action taken 10 
Off-site monitoring 51 
e Remedial action taken 15 
File closed 52 
Payments 
Claims documented 34 
Errors corrected 12 
Payment according to terms and conditions 21 | 
Payment within allowable limits 12 | 

| 
Grants : 2 - = | 

15 


Payment according to eligibility criteria 


Source: Internal Audit Bureau, Human Resources Development Canada 
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In most files, it was 
impossible to 
determine how the 
selection criteria had 
been applied. 


7-10 


Canadian Jobs Strategy, and so the stated 
purpose of the agreement reflected that 
program and not The Atlantic Groundfish 
Strategy. None of these agreement forms 
were modified to reflect TAGS objectives. 


7.27 In some cases, funds provided for 
a project were attributed to a measure 
other than the one under which the 
agreement had been signed. These 
changes were reflected on a schedule to 
the agreement that identified the project’s 
eligible costs, but with no explanations for 
the change. Moreover, the changes were 
not made in the body of the agreement 
that spells out for the sponsor the terms 
and conditions that apply. There were 
unexplained differences between 
agreements, particularly in their clauses 
on objectives, sponsor/co-ordinator/ 
employer eligibility, costs, eligibility of 
participants, recruitment of participants, 
training, payments, GST and disposition 
of assets. 


7.28 Terms and conditions of one 
measure were sometimes confused with 
those of another and applied under the 
wrong agreement. For example, 
agreements under the Delivery Assistance 
measure were used to contract for 
counselling services in remote areas, but 
the correct measure for those services was 
Employment Assistance. The Portable 
Wage Subsidies and Job Opportunities 
measures were similarly confused. 


7.29 Whether it was a case of 
attributing the funds to the wrong measure 
or of using the wrong measure, the result 
was that the wrong terms and conditions 
were communicated to the project 
sponsor. This makes it difficult to 
determine under what criteria the projects 
were approved, managed, controlled and 
monitored. It also puts into question the 
reliability of reports on the funds used for 
each measure. 


7.30 More than half of the agreements 
were signed after the project started. In 
one case, the project officer issued a 


“prior commitment letter” to the sponsor 
before the agreement was approved, 
allowing the project to proceed without a 
formal agreement. In another case, an 
agreement for $141,200 was signed after 
the project had been completed and the 
claim for reimbursement received. In 
addition, payments of over $50,000 were 
made to a few projects with no signed 
agreement. 


73st The auditors could not verify the 
authenticity of some signatures in all 
HRDC regions. Moreover, few files 
contained evidence that the sponsor 
signatories were signing officers with their 
respective organizations. 


7.32 A project established at $463,000 
was split into four separate projects so that 
the local manager could sign the 
agreements. This practice does not comply 
with the delegation of authority. 


7.33 Few projects were subject to 
on-site operational or financial monitoring 
and only about half the files showed 
evidence of any off-site monitoring 
(phone calls, memos and so on). In 
particular, one company received in 
excess of $500,000 over the life of a 
project, went into receivership and was 
closed down before the project was 
completed. There was no evidence that the 
project had been monitored. Indications of 
problems existed early in the life of the 
project, and monitoring could have 
prevented the loss of more than $100,000 
that had been advanced to the sponsor but 
was not accounted for. 


7.34 Some recipients of 
Self-Employment Assistance continued to 
receive payments for several months after 
their businesses had closed down. In 
another case, an employer under a 
Portable Wage Subsidy agreement 
continued receiving payments after the 
employee for whom the contribution was 
intended had become a partner in the 
company. 


1S In most cases, expenditures were 
reimbursed although the claims had been 
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submitted without supporting 
documentation. In some cases, errors in 
claims went undetected and so payments 
were made for expenses that were not 
covered by the agreement. 


7.36 Although most of the audited 
projects had been completed for almost 
two years, nearly half the files showed no 
evidence of the required close-out 
procedures. This made it impossible to 
determine, among other things, if the 
sponsor had accounted for all advances 
received, if overpayments had been made 
and had been recovered, if assets had been 
disposed of as directed, and if the project 
had achieved the expected results. 


hey) In summary, the review of the 
files found serious weaknesses in the 
management, monitoring and control of 
contribution agreements. Proposals did not 
exist or were not sufficiently developed to 
allow proper evaluation. Some agreements 
did not contain complete information, 
included ineligible costs, or did not 
correspond to the measure under which 
the project was funded. Expense claims 
were not supported by documentation, and 
there was very little evidence of on-site 
monitoring visits to examine expense 
records. Objectives and expected results 
of projects were generally not clear. In 
general, the outcome of projects was not 
documented or known. 


Grant Payments Generally 
Complied 


Applicable policies and procedures were 
followed 


7.38 Only two of the labour 
adjustment measures used grants, 
University Tuition and Employment 
Bonus. The audit found that in two cases 
the Employment Bonus was paid prior to 
the completion of the 52-week 
employment period. The employment 
counsellor was responsible for 
determining the eligibility of a participant 
and the amount of the bonus to be paid. 
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In most cases, there was a request from 
the employment counsellor to the project 
officer to initiate payment of the grant and 
some files contained letters of 
understanding that detailed the eligibility 
criteria for the bonus. Few files, however, 
contained the participant’s information 
form and showed how the bonus was 
calculated. 


Conclusion and 
Recommendation 


(Re We concur with the Internal 
Audit Bureau (IAB) that it is not possible 
to certify that most of the audited projects 
funded under various active labour 
adjustment measures met TAGS criteria; 
that they were properly managed, 
controlled and monitored; and that the 
grants and contributions were used for 
their intended purposes. 


7.40 We are concerned about the 
findings of this audit. They clearly 
indicate a lack of diligence in assessing 
proposals and signing agreements, and a 
lack of monitoring. 


7.41 These weaknesses could be a 
reflection of the circumstances that 
prevailed when TAGS was implemented, 
but that does not explain them away. We 
believe that the observations made in this 
chapter could apply to other situations. 
We note, however, that IAB is continuing 
its audit of active grants and contributions 
and has recommended that lessons learned 
from this audit be used to strengthen the 
management of grants and contributions in 
general. 


7.42 Labour adjustment measures 
have been substantially modified. Several 
of the measures reviewed in the audit still 
exist and are applied in a context where 
greater authority has been delegated 
downward in the Department, when not 
wholly vested in the provinces. Although 
TAGS has now been terminated, the 
possibility remains that new partnerships 
may be entered into with the same 
sponsors. 


Few projects were 
subject to on-site 
operational or financial 
monitoring. 


We are concerned 
about the findings of 
the audit conducted by 
the Internal Audit 
Bureau. 
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7.43 Under the current labour 
adjustment programs, Human 
Resources Development Canada should 
ensure that: 


e contribution proposals are 
properly developed and are reviewed 
carefully; 


e analyses and the reasons for 
decisions are clearly documented; and 


e contribution agreements are 
adequately monitored. 


Department’s response: Human Resources 
Development Canada has reviewed the 
chapter produced by the Office of the 
Auditor General. Overall, HRDC agrees 
with the findings; however, the urgency of 
addressing, with constrained resources, 
the immediate needs of the 40,000 TAGS 
clients in the early stages of the 
implementation cannot be understated. 


Where practical, HRDC will address cases 
referred to in the internal audit report that 
require further action. 


HRDC recognizes that the chapter raises a 
number of important points and concerns 
related to the managing, controlling and 
monitoring of TAGS grants and 
contribution agreements. As a result, the 
positive and valuable lessons learned from 
this audit are serving as a basis to assist 
HRDC to better manage, control and 
monitor grants and contribution 
agreements in other programs. In fact, 
HRDC has taken action: to develop new 
policies and procedures, to provide 
training for managers and staff, and to 
secure resources in new initiatives to 
ensure sufficient monitoring. Some of 
these actions are reflected in Chapter § of 
this Report, The Atlantic Groundfish 
Strategy: Follow-up. 
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My About the Audit 


Objective 


The objective of the audit was to respond to a request made by the House of Commons Standing Committee 
on Fisheries and Oceans and to provide assurance that projects funded under TAGS active labour adjustment 
measures had met TAGS criteria; that they had been properly managed, controlled and monitored; and that 
the grants and contributions had been used for their intended purposes. 


Approach 


The Internal Audit Bureau (IAB) of Human Resources Development Canada had planned an audit of grants 
and contributions provided by the Department. We agreed with the IAB that it would conduct the audit, and 
we would monitor and review its work to assess its relevance to our objectives and to determine whether the 
supporting evidence was appropriate for our purposes. 


We expected: 


e that the internal audit work would be planned and conducted in accordance with generally accepted 
auditing standards; 


e that audit reports would properly outline findings, be discussed with management, and be submitted to 
the Deputy Minister or the Audit Committee in a timely manner. 


The-criteria used for the audit by IAB are set out in paragraph 7.20 of the chapter. 


Scope 


Our audit of grants and contributions involved reviewing the work done by the IAB as it evolved. We 
conducted additional reviews where necessary. 


Audit Team 


Assistant Auditor General: David Rattray 
Principal: Louis Lalonde 
Director: Sylvie Paré 


For information, please contact Louis Lalonde. 
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OTTAWA OFFICE CONSTITUENCY OFFICE 
Room 460 niagies 98 Elizabeth Drive 
Confederation Bidg. HOUSE OF COMMONS Gander 
House of Commons OTTAWA, CANADA Newfoundiand 
posi K1A OAG AIV 139 


GEORGE BAKER, M.P. 
Gander-Grand Falls 


October 23, 1997 
OTTAWA 


Mr. L. Denis Desautels, FCA., 
Auditor General of Canada 
240 Sparks Street 

Ottawa, Ontario 

K1A 0G6 


Dear Mr. Desautels: 


During your testimony before the Committee on October 21, 1997, you advised us 
that you have the mandate and the ability to conduct a further, or more in-depth audit, to 
determine “where the money went” in regard to the list of TAGS-funded items provided 
by “The Help Group”: 


Please accept this letter as an official request on behalf of the Steering Committee 
of Fisheries and Oceans to have said audit conducted at your earliest possible 


convenience. 


I would appreciate hearing back from you as soon as possible as to what sort of a 
time frame this audit will involve so that I may report back to the Steering Committee. 


Yours truly, 
wes “f yg ae 


George Baker, M.P., 
Chairman, 
Fisheries and Oceans Committee 


lab/GB 


ive 
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Chapter 8 


The Atlantic Groundfish Strategy 


Follow-up 


The follow-up work was conducted in accordance with the legislative mandate, 
policies and practices of the Office of the Auditor General. These policies and 
practices embrace the standards recommended by the Public Sector Accounting 
and Auditing Board (PSAAB) of the Canadian Institute of Chartered Accountants. 
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8.1 Fishery Restructuring and Adjustment Measures — East Coast 
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The Atlantic Groundfish Strategy 


Follow-up 


Main Points 


8.1 We believe the government’s efforts to implement the recommendations in our October 1997 Report 
Chapter 16 on The Atlantic Groundfish Strategy (TAGS) have been satisfactory. 


8.2 New fishery restructuring and adjustment measures are being implemented. In contrast to our 
observations of 1997, the eligibility criteria for the new measures are clear, logical and applicable. The 
accountability framework established for the measures corrects the shortcomings we had identified in TAGS: it 
clearly defines the responsibilities of the organizations involved, sets out an overall strategic plan and provides for 
a formal co-ordination mechanism. 


Background and other observations 


8.3 The objective of TAGS (1994—98) was to restructure the fishery industry in Atlantic Canada to make it 
economically viable and environmentally sustainable. 


8.4 Close to $1.9 billion was allocated to TAGS. Most of the funding was used to provide income support to 
the some 40,000 fishers and plant workers affected by the groundfish moratorium. 


8.5 In October 1997, we urged the government to carefully examine the impact of the decisions made under 
TAGS in order to benefit from the valuable lessons that could be learned. 


8.6 This follow-up examined progress to date in addressing our 1997 recommendations. Some of our 
observations in this chapter relate to TAGS and others to the Atlantic fishery restructuring and adjustment 
measures announced in June 1998. 


8.7 Human Resources Development Canada completed an evaluation of the labour adjustment component of 
TAGS. It also reviewed the duration of eligibility of TAGS participants. The departments involved in 
implementing the new measures now have considerable information on the profile of the targeted populations, in 
sharp contrast to the situation that prevailed when TAGS was developed. 


8.8 We believe the orderly way in which the departments are implementing the new fishery restructuring and 
adjustment measures represents satisfactory progress. 
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Introduction 


8.9 In our October 1997 Report, we 
urged the government to carefully 
examine the consequences and impacts of 
the decisions made under The Atlantic 
Groundfish Strategy (TAGS) in order to 
benefit from the valuable lessons that 
could be learned. That audit covered the 
measures established by Fisheries and 
Oceans and Human Resources 
Development Canada (HRDC) to account 
for the delivery of TAGS, and the labour 
adjustment component administered by 
HRDC. 


8.10 As our 1997 Report was going to 
press, HRDC initiated consultations on the 
impact that the end of TAGS would have 
on provinces, communities, families and 
individuals in Atlantic Canada and 
Quebec. The results of these consultations 
were presented in the Post-TAGS Review 
Report (Harrigan Report). 


8.11 According to the report: 


One:of the strongest messages from 
the consultations is that government 
policy makers must avoid a 
‘one-size-fits-all’ approach when they 
consider options for the post-TAGS 
environment. The situation differs 
greatly from place to place, 
influenced by such factors as the 
relative strength of the local and 
regional economies or the availability 
of alternative employment, either 
inside or outside the fishery. 


8.12 For its part, the Standing 
Committee on Fisheries and Oceans held a 
series of hearings on fisheries 
management and TAGS. In March 1998, it 
tabled the Baker Report in the House, 
recommending that TAGS be extended to 
May 1999 along with the early retirement 
and voluntary licence retirement programs 
wherever fishing capacity could be 
reduced. Other recommendations 
concerned the creation, with community 
input, of a federally funded infrastructure 
and job diversification program directed 
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toward those most affected by the collapse 
of the Atlantic groundfish fishery. 


8.13 TAGS, which had been aimed at 
restructuring the fishery industry in 
Atlantic Canada to make it economically 
viable and environmentally sustainable, 
ended on 29 August 1998. 


Fishery restructuring and adjustment The Atlantic 
arate” Groundfish Strategy 
8.14 On 19 June 1998, the government ended on 29 August 
announced a series of measures designed 1998 


“to provide individuals with options for 
their future, to help fishing communities 
diversify their economies and to 
encourage and capitalize on opportunities 
in anew economy.” Some of the measures 
apply to communities and individuals not 
only in Atlantic Canada and Quebec but 
also on the West Coast. 


8.15 The objectives of the 
restructuring and adjustment measures are 
to: 


e bring closure to TAGS; 


e continue the restructuring of the 
Atlantic groundfish fishery; 


e conserve and protect West Coast 
salmon stocks; 


e assist individuals and communities 
with adjustment; and 


e ensure balance and equity between 
regions. 


The government has said that these 
objectives are to be achieved in 
consultation with provincial governments, 
where required. 


8.16 A total of $760 million will be 
allocated to implement the measures on 
the East Coast. At the time of the 
announcement, the Minister of Fisheries 
and Oceans indicated that Atlantic 
groundfish stocks would take many years 
— possibly decades — to rebuild, and that 
even then the sector would not be able to 
support the same number of fishers and 
fishery workers. 
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8.17 The measures offer a variety of Focus of the follow-up 
initiatives to help people who permanently 
exit the fishery prepare for new 
opportunities. They focus on long-term 
strategies for human resource and 
community economic development that 
equip individuals and communities with 
the skills and assistance they need to 
prepare for life beyond the fishery. 

Exhibit 8.1 lists the measures that apply to 
the East Coast and shows the funds 


allocated to each over several years. 8.21 Two of our recommendations 
applied directly to TAGS and four others 
looked to the future. This follow-up 
focussed on action related to the new 
fishery restructuring and adjustment 
measures. Further details on the scope and 
criteria of our work are provided at the 
end of the chapter, in About the 
Follow-up. 


8.20 The purpose of the follow-up was 
to assess the progress made in addressing 
the observations and recommendations in 
our October 1997 Report Chapter 16 on 
The Atlantic Groundfish Strategy. HRDC 
agreed with our recommendations and, in 
August 1998, submitted a report to the 
Standing Committee on Public Accounts 
on the actions it had taken. 


8.18 When the government announced 
the measures, it stressed that they were to 
be finalized in consultation with provinces 
and stakeholders. It emphasized that the 
federal government alone could not 
address these issues: governments, 
industry, unions and individuals would 
have to work together to find lasting 
solutions to these challenges. 


8.19 Overall, the new fishery Observations 


restructuring and adjustment measures are 

similar to the initiatives offered under TAGS Recommendations 

TAGS. However, they do differ in some Addressed 

respects: they take into account 

individuals and communities on the West A program evaluation of good quality 
Coast as well; there is no ongoing income was conducted 

support; participation of eligible 


individuals is voluntary; and they are not 8.22 In our October 1997 chapter, we 
linked by a common objective — each recommended that HDRC continue its 
measure has its own objective. efforts to evaluate the labour adjustment 


component of TAGS and report on the 
results, thus contributing to an enlightened 
discussion of all programs of this type that 
might be considered in the future. 


Exhibit 8.1 8.23 The Department completed its 

; evaluation and released its final report in 
Fishery Restructuring and Adjustment Measures April 1998. We found that for the most 
- East Coast 


part, the evaluation asked the right 


: questions and formulated them 
Funding ie : 

Rasaaece ($ millions) appropriately. The evaluation also 
—— — provided valuable answers to those 

Licence retirement 250.0 questions. Some observations supported 
| Final cash payment — TAGS income support 164.5 elements of TAGS and others were critical 

of its results, objectives and 
Adjustment measures 150.5 implementation. The evaluation made 
Economic development measures 100.0 several observations that could be useful 
to the Department in the development of 

Early retirement 95.0 

Eee = : : £ new programs. 
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Duration of eligibility of TAGS 
participants was reviewed 


8.24 We recommended that HRDC 
ensure that the review of eligibility of 
TAGS participants in the Newfoundland, 
Prince Edward Island and Nova Scotia 
regions be completed as soon as possible; 
and that it consider whether the review 
should be extended to all TAGS 
participants. The purpose of that 
recommendation was to ensure, through a 
uniform review of the supporting 
documentation, that the eligibility criteria 
had been applied equitably. 


8.25 The Department agreed with the 
recommendation and undertook to 
complete the review of eligibility files for 
all regions covered by TAGS. The files 
reviewed were those of participants whose 
duration of eligibility was less than five 
years, the group in which errors were most 
likely to have occurred. Fifty-two percent 
of some 40,000 TAGS participants fell 
into this category. 


8.26 At the time of our follow-up, 

97 percent of the files identified for 
review had been completed. We are 
satisfied with the rigour of the review, and 
the results confirm that there had been a 
very high rate of error. 


Improvements Evident 


Better information was used in 
developing the new measures 


8.27 We had also recommended that 
the government ensure that for initiatives 
of this magnitude, departments have 
enough time to develop strategies that 
take into account completed analyses and 
possible options. In its August 1998 report 
to the Public Accounts Committee the 
Department stated: 


The Post-TAGS Review Report, along 
with other reports and studies, such as 
the Auditor General’s Report, The 
East Coast (Baker) Report, work done 
on the Employment Insurance Act 
Part II, the Formative Evaluation of 
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TAGS and the Final Evaluation of 
TAGS, served as a basis in 
formulating possible options to 
address the post-TAGS situation. 


8.28 In May 1997, the government 
announced that it would examine the 
impact of the end of TAGS on individuals, 
communities and provinces. The 
Post-TAGS Review Report (Harrigan 
Report), released in January 1998, 
presented the results of that review. The 
consultations with interested parties that 
were part of the review provided 
indications of not only the needs to be 
addressed but also possible directions and 
options. 


8.29 Further, in October 1997 HDRC 
conducted an analysis of the population 
affected by TAGS, particularly those who 
would still be eligible when it ended. The 
Post-TAGS Review Report presented the 
results of that analysis as well. 


8.30 The departments now have access 
to more specific data for establishing the 
profile of target clients, which was not the 
case when TAGS was being developed. In 
fact, as shown by the analyses conducted 
for the Post-TAGS Review Report, it is 
possible to establish precisely the number 
of individuals who may be eligible for 
different measures, their place of 
residence, level of education, age and 
family status. The various studies that 
were conducted, including those for the 
program evaluation, have contributed to 
this improved information and made 
possible more accurate forecasts. 


8.31 Our review of the supporting 
documents indicated that the new 
measures were developed on the basis of 
completed analyses and took into account 
various options. 


The new measures were launched in an 
orderly way 


8.32 We had recommended that the 
government ensure that any future strategy 
of this magnitude be launched in a 
systematic way. In its response to the 
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The files reviewed 
were those of 
participants whose 
duration of eligibility 
was less than five 
years, the group in 
which errors were 
most likely to have 
occurred. 
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Public Accounts Committee, the 
Department indicated that in establishing 
the new fishery restructuring and 
adjustment measures it had considered the 
lessons learned from the implementation 
of TAGS. 


8.33 It pointed out that 
interdepartmental meetings are being held 
on a regular basis, thereby ensuring a 
co-ordinated, systematic and consistent 
approach to the implementation of the 
measures. An operational plan had also 
been developed that outlined key 
activities, indicated who would be 
responsible for each, and provided 
specific time frames. We verified that this 
response was accurate. 


8.34 Consultations on several of the 
measures announced last June were held 
with the provinces, unions and other 
stakeholders before the measures were 
finalized, and their details were not 
announced until the end of July. 
Negotiations on the early retirement 
package are ongoing, as this is a 
federal-provincial cost-shared program. At 
this writing, agreements had been signed 
with Newfoundland, Nova Scotia and 
Prince Edward Island. 


8.35 The adjustment measures 
comprise several components, including 
self-employment, new skills development, 
wage subsidies, term job creation and 
mobility assistance. Delivery of the 
employment benefit and support measures 
under Part II of the Employment Insurance 
Act has been devolved to two of the 

five provinces. Agreements have to be 
negotiated to offer these services (except 
for the term job creation component, 
which is delivered by the federal 
government) to clients who are eligible for 
employment insurance. Negotiations with 
one of those provinces is under way. 


8.36 The fact that the various 
measures are to be implemented in 
sequence could be seen as limiting the 
choices available to clients. However, our 
review found that suitable mechanisms 


have been established to avoid that 
possibility. 


8.37 As part of our follow-up, we 
visited the Newfoundland Region and 
some Human Resource Centres of Canada 
to determine whether staff had adequate 
information and tools to implement the 
new measures. We observed that the staff 
had access to clear information and to 
guidelines. These offices, like 
headquarters, are following an operational 
plan that outlines the activities, indicates 
who is responsible for each, and provides 
specific time frames. This plan is closely 
followed and updated regularly. 


8.38 In our view, the implementation 
of measures under HRDC’s responsibility 
is moving forward in an orderly way. 


Eligibility criteria are tailored to the 
measures 


8.39 In 1997, we recommended that 
given the problems encountered in 
applying TAGS eligibility criteria, future 
initiatives use criteria that are clear, 
logical and applicable. 


8.40 In its response to the Public 
Accounts Committee, the Department 
stated: 


To be eligible for the newly 
announced fishery restructuring and 
adjustment measures for the Atlantic 
groundfish industry, you must have 
been eligible for TAGS. This 
information currently exists in TAGS 
systems. In addition, other than for 
the early retirement measure, 
individuals will not be required to 
complete an application to access the 
measures offered, nor will there be 
any need for an assessment of 
eligibility to be undertaken. 


Since then, a number of criteria have been 
modified or improved as a result of the 
consultations with the provinces and 
various stakeholders. 


8.41 The measures address different 
populations. For example, the final cash 
payment applies only to individuals who 
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were eligible for TAGS on 

29 August 1998. The early retirement 
measure is available only to individuals 
who were between the ages of 55 and 64 
on 31 December 1998 and were eligible 
for TAGS on | January 1998. The licence 
retirement measure is open to groundfish 
licence holders who have vessels under 
65 feet. Priority will be given to core 
fishers whom Fisheries and Oceans had 
identified as part of the fishery of the 
future, and to TAGS recipients. However, 
recipients who are eligible for these 
measures may choose only one of them. 


8.42 Mobility assistance on the East 
Coast is available to eligible TAGS 
participants, including those whose 
benefits ran out before 29 August 1998. 
Individuals who took early retirement, 
agreed to retire their licence or received 
an employment bonus under TAGS are not 
eligible. Nor are those who have already 
moved to anew community outside the 
five regions covered by TAGS. Clients 
who have taken early retirement under the 
new fishery restructuring and adjustment 
measures are also excluded. 


8.43 Priority for term job creation in 
the East Coast is given to former TAGS 
clients. This component 1s also offered to 
unemployed residents of the communities 
affected by the collapse of the Atlantic 
groundfish fishery. In participating, the 
target population will obtain insurable 
employment that will assist them to 
become eligible for employment insurance 
and therefore qualify for the benefits and 
support measures offered under the 
Employment Insurance program. 


8.44 To be eligible for self- 
employment, skills development and wage 
subsidies, individuals must be former 
TAGS participants. Those who opted for 
early retirement, licence retirement or the 
employment bonus under TAGS or early 
retirement under the fishery restructuring 
and adjustment measures are not eligible. 


8.45 The information required to 
evaluate eligibility for these various 
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measures is accessible through the 
systems of the departments involved. 
Clear, logical and applicable criteria have 
been established. 


Formal measures have been established 
to ensure accountability 


8.46 In 1997, we observed major 
weaknesses in the accountability 
framework for TAGS that had an impact 
on value received for the money 
expended. Some of these weaknesses were 
as follows: the responsibilities of the 
organizations charged with developing 
and implementing TAGS were not clearly 
defined and agreed upon in a 
memorandum of understanding; no 
integrated strategic plan was developed; 
there was no formal mechanism for 
co-ordinating activities; and none of the 
information presented to Parliament made 
it possible to determine the progress made 
toward achieving the objectives of TAGS. 
We recommended that in any future 
strategy involving a number of federal 
organizations the government institute 
formal measures to ensure accountability 
for the strategy as a whole. 


8.47 In its response to the Public 
Accounts Committee, HRDC noted that 
work had begun on drafting a 
memorandum of understanding (MOU) 
among the departments and organizations 
responsible for implementing the Atlantic 
and West Coast fishery restructuring and 
adjustment measures. The MOU has now 
been signed by all parties. 


8.48 The parties to the MOU are 
Fisheries and Oceans, Human Resources 
Development Canada, the Atlantic Canada 
Opportunities Agency, Canada Economic 
Development for Quebec Regions, 
Western Economic Diversification Canada 
and Indian and Northern Affairs Canada. 


8.49 The MOU sets out the roles and 
responsibilities of each federal department 
and agency. Its objective is to provide a 
framework for the departments to 
co-ordinate their efforts, while 
recognizing the specific responsibilities of 


Clear, logical and 
applicable criteria 
have been established. 
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Sufficient information 
was available to 
develop the new 
measures. 


each in implementing the individual 
measures. The MOU provides for the 
creation of a Steering Committee to serve 
as the formal co-ordination mechanism, 
and regional committees. 


8.50 The MOU provides for an 
implementation schedule, a financial 
management framework, an accountability 
framework, exchange of information and 
evaluation of the measures’ results. It is 
too early, however, to assess whether all of 
its aspects will work. 


Conclusion 


8.51 Our review established that 
sufficient information was available to 
develop the new measures. The review of 
the impact of the end of TAGS, the Baker 
Report and the studies by the Program 
Evaluation Directorate provided useful 


information to identify options, analyze 
their potential impacts and make choices. 


8.52 Our review of the eligibility 
criteria adopted for the various measures 
found that they were clear, logical and 
applicable. We also determined that 
HRDC had thoroughly reviewed the 
eligibility of TAGS participants with a 
duration of benefits under five years and 
that all cases were dealt with on a 
consistent basis. 


8.53 ~ Particular attention has been 
given to the launching of the new 
measures. The proposed accountability 
framework for the various measures is 
appropriate. Thus, in our opinion, efforts 
so far to avoid the uncertainty that 
surrounded TAGS have been satisfactory. 
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yy About the Follow-up 


Objective and Criteria 

Our objective was to follow up on the progress made in response to the following recommendations as set out 

in our October 1997 Report Chapter 16 on The Atlantic Groundfish Strategy (TAGS): 

e Incontemplating any future initiatives of TAGS magnitude, the departments should have enough time to 
develop strategies that take into account completed analyses and possible options. 

e Future strategies of TAGS magnitude should be launched in a systematic way. 

e Eligibility criteria for any similar strategy should be clear, logical and applicable. 

e The review of the duration of eligibility for TAGS should be completed as soon as possible. 


e Human Resources Development Canada (HRDC) should evaluate the labour adjustment component of 
TAGS and report the results. 


e Formal measures should be put in place to ensure overall accountability for any similar strategy. 


Scope 


Our follow-up consisted of a review of the action plan submitted by HRDC to the Standing Committee on 
Public Accounts in response to our 1997 observations and recommendations. We interviewed some 
government officials and reviewed supporting documentation. The follow-up was carried out primarily at 
HRDC headquarters and in its Newfoundland region as well as at Fisheries and Oceans headquarters. 


Audit Team 


Assistant Auditor General: David Rattray 
Principal: Louis Lalonde 
Director: Sylvie Paré 


Andrée Bélair 
Martin Dompierre 


For information, please contact Louis Lalonde. 
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Follow-up 


The follow-up work was conducted in accordance with the legislative mandate, 
policies and practices of the Office of the Auditor General. These policies and 
practices embrace the standards recommended by the Public Sector Accounting 
and Auditing Board (PSAAB) of the Canadian Institute of Chartered Accountants. 
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Management of Science and 
Technology Personnel 


Follow-up 


Main Points 


94 We are satisfied with the efforts made by the science and technology community to follow up on our 
audit recommendations of 1994 and the concerns we raised in our 1996 follow-up report. In our view, the 
community is showing leadership and perseverance in dealing with the human resource management issues we 
raised. 


92 As a result of all the work done since 1994, the science and technology community is now in a position to 
act. But the community is faced with considerable challenges. It must give priority to resolving the oncoming 
changes in its demographic profile that could weaken or compromise the government’s science and technology 
capacity. It must tackle the dual challenges of attracting and recruiting promising young scientists and 
technologists while retaining high-calibre employees to mentor and develop the new recruits. The community is 
thus seeking new measures, tools and resources for external recruitment, as it estimates that over the next five 
years it may have to recruit between 2,500 and 3,300 employees to build a renewed and rejuvenated science and 
technology work force. 


Background and other observations 


a3 At 31 March 1998, the federal government had close to 20,000 scientific and technical employees 
working in science-based departments, agencies, Crown corporations and research establishments in the fields of 
natural sciences and engineering (hereinafter referred to as the “science and technology community”). 


9.4 The science and technology community makes an important contribution to the government’s 1996 
science and technology strategy, which focusses on sustainable job creation and economic growth, improved 
quality of life and the advancement of knowledge. 


a Expenditure and work force reduction in the public service has changed the profile of the science and 
technology community and worsened the long-identified problems of rejuvenation and recruitment. The change in 
the age profile is a major challenge to the future of the community. Not only have most senior and experienced 
scientists and technologists left the government since 1994, but the youngest and most promising as well. 


9.6 Following our 1994 audit of federal science and technology activities, the community mobilized to 
develop a management framework and a results-oriented plan for human resources management in science and 
technology. It addressed such issues as the need for a more strategic approach to the management of scientific 
personnel; for more systematic renewal of scientific personnel; and for more effort to maintain the skills and 
knowledge base in research establishments. Working groups were created to study important human resource 
issues. Among their recommendations to the Science and Technology Senior Steering Committee on Human 
Resources was that new mechanisms be adopted and human resource strategies developed to improve the 
management of science and technology personnel in science-based departments and agencies. 
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ay In 1994, we pointed out the need to develop a stronger and more effective management capability. Since 
then, the community has developed a competency profile for science and technology managers. During our 
consultations, we noted that most science-based departments used their own competency profiles instead of the 
one developed by the working group. Moreover, their profiles were being used solely to identify training needs 
and generally not for purposes of manager recruitment, promotion or performance assessment. Present practices 
suggest a lack of consensus in the community on the management competency profile defined by the working 
group. This could eventually prevent the integration of recruitment and training activities as well as the reward, 
promotion and compensation systems envisioned in the Science and Technology Blueprint for Human Resources 
Management. 


9.8 The Treasury Board Secretariat, science-based departments and agencies and the science and technology 
community have indicated that they are committed to following through on the strategies and plans developed to 
date. 
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Introduction 


9.9 The federal government spends 
approximately $6 billion per year, or a 
quarter of the total investment in 
Canadian research and development, 
making it a leading stakeholder in the 
country’s general science and technology 
activities. In 1996 the government 
released its science and technology 
strategy, which focussed on three 
objectives: job creation and economic 
growth; improved quality of life for 
Canadians; and the advancement of 
knowledge. 


9.10 At 31 March 1998, an estimated 
20,000 federal employees were involved 
in scientific and technological activities, 
in the fields of natural sciences and 
engineering (hereinafter referred to as the 
“science and technology community” or 
“community’”’). These employees worked 
in the six major science-based 
departments — Agriculture and Agri-Food 
Canada,. National Defence (civilian 
personnel), Environment Canada, 
Fisheries and Oceans, Natural Resources 
Canada and Health Canada, for whom 
Treasury Board is the Employer — and in 
science-based agencies and Crown 
corporations such as the National 
Research Council and the 
Communications Research Centre. 


9.11 There have been many attempts 
over the past 30 years to resolve some of 
the major issues in the management of 
scientific personnel, particularly 
management capability and the 
maintenance of a “world-class” scientific 
work force. In 1994, following our audit 
of federal science and technology 
activities, the science and technology 
community mobilized and developed a 
better-structured approach to human 
resource management. In March 1996, the 
Treasury Board Secretariat released the 
Framework for the Human Resources 
Management of the Federal Science and 


Management of Science and Technology Personnel: Follow-up 


Technology Community. Among other 
things, this document outlines objectives 
and the management structure put in place 
to address human resource issues, notably 
those raised in our 1994 Report 

(Chapter 11). 


9.12 In our 1996 follow-up 
(Chapter 15) we noted that the Treasury 
Board Secretariat, in co-operation with 
science-based departments and agencies 
and the science and technology 
community, had undertaken a series of 
initiatives designed to address our 1994 
audit recommendations. We pointed out, 


The science and 
technology community 
had undertaken a 
series of initiatives 


however, that the litmus test of the designed to address 

Framework would be the degree to which our 1994 audit 

the government accepted it and the : 
recommendations. 


science and technology community 
implemented it. 


9.13 It is important to note that this 
series of initiatives was undertaken in a 
particularly difficult environment, when 
wages and salaries were frozen at their 
1991 levels and the government was 
starting work on its “Getting Government 
Right” initiative by launching a number of 
reviews. One of these was Program 
Review — the scrutiny and review of 
“program spending” and the 
re-examination of the federal 
government’s role and responsibilities in 
delivering programs. Program Review led 
to substantial expenditure and work force 
reductions in the public service. All of this 
had a significant impact on the activities 
and capacities of science-based 
departments and on the morale and job 
satisfaction of their employees. 


This series of 
initiatives was 
undertaken in a 
particularly difficult 
environment. 


Focus of the follow-up 


9.14 Given the importance of the 
Framework for the Human Resources 
Management of the Federal Science and 
Technology Community and the fact that a 
little more than two years had passed 
since our last follow-up, we believed that 
the time was right to review the progress 
made in implementing the Framework. 
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The community’s 
management Blueprint 
responds in essence 
to our 1994 
recommendations on 
developing a strategy 
for the management of 
scientific personnel. 


This chapter provides our assessment of 
advances since 1994 in: 


e addressing the problems of work 
force renewal, rejuvenation and 
recruitment of scientific personnel; and 


e improving management capability by 
developing competency profiles for 
managers of research establishments. 


9.15 We interviewed some 
stakeholders in the science and technology 
community and consulted various reports 
published by the community and others. 
We used the Treasury Board Secretariat’s 
information systems to quantify certain 
results. Details concerning the objective, 
scope and approach of our work are 
included in About the Follow-Up at the 
end of this chapter. 


Observations 


Many Activities Address 
Problems Raised in 1994 


The community is developing a 
management blueprint and 
accountability agreements 


9.16 In 1994 we were concerned by 
the government’s lack of commitment to 
implementing the recommendations of 
studies done over the preceding 30 years, 
and by the fact that there was no one 
responsible for taking corrective 
measures. In 1996 we recognized that the 
support of the Treasury Board Secretariat, 
science-based departments and agencies 
and the science and technology 
community had contributed much to 
advance the implementation of the 
Framework for the Human Resources 
Management of the Federal Science and 
Technology Community. We also pointed 
out, however, that it would require a major 
effort by science-based departments and 
agencies to carry out the activities 
identified in the Framework. In that 
context, we raised the importance of 
results-oriented, time-phased plans that 
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would include main steps, schedules, 
milestones and resource requirements. 


9.17 On the community’s behalf, in 
the summer of 1997 the Treasury Board 
Secretariat provided us with a 
management document, Science and 
Technology Blueprint for Human 
Resources Management. 


9.18 The Blueprint was developed in 
co-operation with all the stakeholders — 
namely the Secretariat, science-based 
departments and agencies, and the unions. 
It sets out action plans and expected 
results, implementation activities and pilot 
projects. It identifies responsibilities, 
particularly those of the working groups 
looking at the human resource 
management issues raised in our 1994 
audit. The Blueprint suggests priority 
actions and a timetable for 
implementation as well as the necessary 
resources. Finally, it presents success 
criteria, performance measures for the 
expected results, and mechanisms to 
ensure accountability for results. 


9.19 In our opinion, the Blueprint 
responds in essence to our 1994 
recommendations on developing a 
strategy for the management of scientific 
personnel. It offers a particular benefit as 
a guide to implement human resource 
initiatives and to integrate and co-ordinate 
horizontal science and technology issues. 
Moreover, the Blueprint contains enough 
information to enable science-based 
departments and agencies to carry out the 
activities needed for improved 
management of their scientific personnel. 
The accountability structure 1s noteworthy 
and the community has established what 
we believe are realistic expectations for 
meeting the short-term objectives. 


9.20 The Blueprint will continue to 
offer these benefits if its designers and 
users maintain the consultative approach 
used so far to resolve current and 
emerging issues in the management of 
scientific personnel. Moreover, we believe 
that the Blueprint, regularly updated, can 
keep the science and technology 
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community informed about progress made 
and results achieved. 


9.21 In the course of our follow-up, 
we noted that the working groups have 
reported on the status of their work to the 
Science and Technology Senior Steering 
Committee on Human Resources. It, in 
turn, has presented important human 
resource issues to the Science and 
Technology Community Sub-Committee 
of the Committee of Senior Officials (see 
Appendix). The community Internet site 
(http://www.tbs-sct.gc.ca/tb/hr/scitech) 1s 
managed by the Treasury Board 
Secretariat and posts, among other things, 
reports on the results of the working 
groups’ efforts. 


9.22 In our opinion, the accountability 
structure could be still more transparent 
by identifying, for example, how progress 
and results are to be reported to 
Parliament and the community. In 1998, 
the science and technology community 
presented its findings in the First Progress 
Report on La Reléve. (Introduced in 1997, 
La Reléve is a government initiative 
aimed primarily at modernizing the public 
service of Canada and making full use of 
its talents.) At the end of our follow-up 
work, the community was discussing the 
nature and the content of a progress report 
for 1999 and future years. 


The community discusses issues of 
common interest 


9.23 In 1994, we recommended that 
the government create a forum dedicated 
to scientific personnel management issues. 
At the end of our follow-up work in 1996, 
we were still concerned about the lack of 
a forum for sharing “best practices” in 
human resource management and 
discussing issues of common interest. 


9.24 In December 1998, the 
community held a forum for science 
managers on the theme “The Science and 
Technology Workforce — Managing Your 
Investment”. The Treasury Board 
Secretariat, science-based departments 
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and agencies, unions and the community 
discussed best practices and innovative 
solutions to current and future human 
resource issues, such as the management 
of tomorrow’s science and technology 
personnel. 


9.25 This forum was an opportunity to 
discuss issues of common interest and 
propose solutions to identified problems. 
To obtain the commitment and continued 
support of the community and other 
stakeholders, however, attention will have 
to focus on consultation and 
communication as well as on achieving 
expected results. When scientific 
managers, researchers and other 
stakeholders collaborate to identify their 
own human resource problems, they will 
be more inclined to adopt the proposed 
solutions and to maintain their 
commitment. The science and technology 
community continues to make 
communication a priority for 1999. 


The Working Groups Are 
Recommending Action Plans 


9.26 In 1996, we underscored the 
amount of time and effort invested by the 
science and technology community in 
addressing long-standing problems raised 
in our 1994 audit. These included the need 
for a more strategic approach to the 
management of scientific personnel; for a 
more systematic process of renewal; and 
for more effort to maintain the skills and 
knowledge base in research 
establishments. We were concerned that 
effort and momentum would dissipate 
because, among other things, major 
initiatives would have to be undertaken at 
the same time as Program Review and 
expenditure and work force reductions. 


9.27 In this follow-up, we noted that 
several mechanisms had been proposed to 
ensure continuity of these initiatives. For 
example, the working group looking at 
work force mobility issues recommended 
to the Senior Steering Committee that the 
science and technology community be 
managed centrally, on the basis of its own 
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All working groups 
submitted reports that 
identified effective 
means, essential 
conditions and action 
plans for improving 
the management of 


particular characteristics and needs. The 
working group added that this centralized 
management system would include an 
infrastructure at the Treasury Board 
Secretariat to provide general support to 
the science-based departments and the 
community. 


9.28 We noted that despite budget cuts 
and work force reduction, the working 
groups maintained continuity of effort 
notwithstanding a few delays. All working 
groups submitted reports that identified 
effective means, essential conditions and 
action plans for improving the 
management of scientific personnel. Their 


9.29 For example, the working group 
looking at recruitment and renewal issues 
recommended to the Senior Steering 
Committee that science-based 
departments and agencies develop and 
implement promotional programs 
designed to raise awareness among 
prospective job candidates about federal 
activities and successes in science and 
technology. The working group added that 
those programs should target students and 
encourage them to consider science as a 
field of study and science-based 
departments and agencies as potential 
employers. Exhibit 9.1 presents some of 
the recommendations made by the 


scientific personnel. 


recommendations touched on a specific 
infrastructure for the science and 


working groups on work force and 
mobility and recruitment and 
rejuvenation. 


technology community, the adoption of 


new mechanisms, the need to develop 9.30 
human resource strategies and the 
elimination of systemic constraints. 


Exhibit 9.1 


The working group looking at job 
classification and salary compression is on 
hold, pending decisions on the Universal 


Some Working Group Recommendations to the Science and Technology Senior Steering Committee on Human Resources 


Working Group 
on Work Force and Mobility 


Working Group 
on Recruitment and Rejuvenation 


e Conduct an analysis and comprehensive projection of the 
science and technology work force across the federal 
government in support of management of the community and 
the initiatives taken by each department. 


e Establish the mechanisms, processes and databases needed to 
implement the sharing of best practices and information 
among science-based departments across the federal 
government. 

e Increase mobility within the federal government by 
completely eliminating the obstacles that currently stand in 
the way of staff movement. 

e Maximize the use of existing official mobility and 
rejuvenation programs by integrating into program design and 
delivery a specific component for science and technology. 


e To complement existing programs, create an 
assignment-based exchange program specifically for the 
science and technology community with the objective of 
offering short- and medium-term interdepartmental 
assignments. The program should also focus on the exchange 
of rotating assignments between the regions and headquarters. 


All science- and technology-based departments and agencies should 
develop a long-term recruitment and rejuvenation strategy. 


Federal departments and agencies should encourage the hiring of 
undergraduate and graduate students as well as research fellows by 
science and technology organizations. 


The science- and technology-based departments and agencies 
should negotiate co-operation agreements with teaching 
establishments, industry and other government bodies in Canada and 
abroad. 


All science- and technology-based departments and agencies should 
adopt recruitment and rejuvenation plans for science and technology 
managers. 

More authority should be delegated to all science- and 
technology-based departments and agencies, along with greater 
leeway in the areas of staffing and recruitment. 


Source: Excerpts from reports of these two working groups to the Senior Steering Committee, 1997. 
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Classification Standard that the 
government intends to implement in 1999, 
The community will have to reconcile the 
job classification and evaluation system 
with a promotion system for research 
scientists that is based on individual 
competencies and contributions. 


931 In our opinion, the necessary 
elements of the planning phase are in 
place; the science and technology 
community is now in a position to act. 


The Challenges of Recruitment 
and Retention Are Considerable 


It is time to act 


9.32 In 1994 we emphasized that 
renewal of the scientific work force 1s 
essential if a scientific organization is to 
remain creative and productive in the long 
term. 


35 In 1996, we found that work 
force reduction had meant the loss of a 
significant number of experienced 
scientists. We noted as well that young 
scientists who were considered promising 
by departments but who were term 
employees had been released to protect 
the positions of indeterminate employees. 
As a result, we concluded that the 
challenge of renewing and recruiting 
scientific personnel was even greater than 
the community had faced in 1994. 


9.34 Since then, we have performed 
other demographic analyses that — along 
with those by the community — show that 
the six major science-based departments 
for whom Treasury Board is the Employer 
saw a net reduction of almost 25 percent 
in their work force between 1994 and 
1998, a decrease of almost 5,000 
employees in all forms of employment. 
We note that around 1,300 of those 
employees were transferred to the 
Canadian Food Inspection Agency. (As a 
separate employer, the Agency and not the 
Treasury Board is the Employer of its 
human resources.) 
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935 Our analysis showed a sharp 
decrease in some occupational groups 
since 1994 (see Exhibit 9.2). Additional 
analysis also showed that indeterminate 
positions at entry levels were hit hardest. 


9.36 The ratio of indeterminate staff to 
term employees dropped. In 1994, the 
number of term employees in the six 
science-based departments represented 

9.5 percent of the population; in 1998, this 
had increased to 16.5 percent. 


9:37 The change in the age profile of 
the indeterminate work force is a major 
challenge for the future of the science and 
technology community and its renewal. 
Our analysis showed that it is the young 
and the oldest who have left the 
community since 1994. From 1994 to 
1998 there was a greater concentration of 
scientific personnel in the middle age 
groups. The number of indeterminate 
employees between the ages of 19 and 40 
decreased by 10 percent, while the 
population over 41 increased by 10 
percent. In 1998, the average age of term 
employees was 36; our analysis showed 
there had been no significant fluctuation 
in age among term employees between 
1994 and 1998. 


9.38 Some of our audit work and 
analyses and the analysis conducted 
recently by the science-based departments 
show similar demographic losses that 
could weaken their operational capacity. 
The problems of rejuvenation, recruitment 
and retention have worsened since 1994; 
resolving them must be made a priority. 
Without vigorous action, the government’s 
science and technology capacity could be 
seriously compromised. Based on its own 
analysis, the community estimates that the 
six science-based departments may have 
to recruit between 2,500 and 3,300 
indeterminate employees over the next 
five years to replace those who will retire 
or leave for other reasons. A working 
group has looked at the needs of the 
science-based departments in the short, 
medium and long terms. 


The necessary 
elements of the 
planning phase are in 
place; the community 
is now in a position to 
act. 


The problems of 
rejuvenation, 
recruitment and 
retention have 
worsened since 1994; 
resolving them must 
be made a priority. 


Without vigorous 
action, the 
government’s science 
and technology 
capacity could be 
seriously 
compromised. 
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The community needs to focus on more undergraduates and graduates than 
recruitment activities they did in the past. The strategies also 
suggest inviting retired scientists to 
promote science at colleges and 
universities and to become mentors for 


9.39 The recruitment strategies of 
science-based departments deal with 


common issues, particularly the need to : 
change the perception of the federal new recruits. At the last Assistant Deputy 
public service as an employer and to Ministers Forum in June 1998, it was 


further clarify the government’s role in the proposed that a co-op program for 


field of science. To promote and students be offered and university 
demystify science, the science-based scholarships be directed toward specific 
departments and agencies plan to recruit priority sectors. 


Exhibit 9.2 


Distribution of Science and Technology Occupational Groups in Six Science-Based Departments! 


All forms of employment 


Net Decrease 
April March or 

Occupational Groups 1994 1998 Increase? CFIA? 
Technical 

Drafting and Illustration (DD) 682 373 (309) 1 
Engineering and Scientific Support (EG) 6,008 4,664 (1,344) 280 
Electronics (EL) 677 722 45 

General Technical (GT) 2,056 1eaT9 (737) 
Scientific 

Agriculture (AG) 283 85 (248) 187 
Biological Sciences (BI) 1,308 INS IIe? (136) 95 
Chemistry (CH) 460, 362 (98) 62 
Defence Scientific Service (DS) 569 439 (130) 

Engineering and Land Survey (EN) 1,224 1,036 (188) 2) 
Forestry (FO) 167 87 (80) 

Medicine (MD) 159 129 (30) 

Meteorology (MT) 621 486 (135) 

Nursing (NU) 1,025 780 (245) 

Physical Sciences (PC) 232. 780, (452) 

Pharmacy (PH) 38 D3 (15) 

Scientific Research: 50 
Scientific Research Manager (SE-REM) 196 165 (31) 

Scientific Research (SE—RES) 1,945 1,527 (418) 

Scientific Regulations and Patents (SG) 438 293 (145) 216 
Veterinary Medicine (VM) 587 33 (554) 423 
Total 19,675 14,425 (5,250) 1,316 


' Agriculture and Agri-Food Canada, National Defence (civilian personnel), Environment Canada, Fisheries and 
Oceans, Natural Resources Canada and Health Canada. 

2 Net decrease or increase accounts for recruitment and departures during the given period. 

3 Positions transferred to the Canadian Food Inspection Agency in 1998. 


Source: Treasury Board Secretariat, 1994 and 1998 Incumbent files (not audited). 
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9.40 The recruitment strategies 
envisioned by the science and technology 
community are in line with the 
recommendations in the recent study by 
the Public Service Commission, Facing 
the Challenge — Recruiting the Next 
Generation of University Graduates for 
the Public Service. The students who 
responded to a survey by the Commission 
showed little awareness of employment 
opportunities in the federal public service 
related to their field of study. 


The community wants more flexible 
external recruitment measures 


9.41 At the December 1998 forum, 
some science managers contended that the 
authority currently delegated to them 
allows them to hire only casual employees 
except in cases where there are special 
agreements with the Public Service 
Commission. 


9.42 As one of the pilot projects 
proposed by science-based departments 
and the community, the Public Service 
Commission in 1997 adopted more 
flexible recruitment measures for the 
Minister of Agriculture and Agri-Food, so 
the Department could meet its priorities 
for co-operation with the private sector 
under the cost-sharing program for 
research and development spending. The 
Commission used an exclusion order, 
which exempts some appointments from 
the Public Service Employment Act in part 
or in whole. (Exclusion orders are 
recommended by the Public Service 
Commission and are approved by the 
Governor in Council.) Progress reports 
show that recruitment for that project took 
an average of only three days. In addition, 
the same exclusion order enabled two 
research establishments to recruit almost 
200 term employees for scientific 
positions between July 1997 and 
December 1998. Eventually some of these 
term employees were offered 
indeterminate employment through the 
regular staffing process. 
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9.43 The science-based departments 
and some agencies are exploring other 
special agreements with the Public 
Service Commission to make external 
recruitment more flexible. In our opinion, 
new external recruitment measures are 
needed to build a renewed and rejuvenated 
work force and thereby resolve the 
problems we have reported since 1994. 
The science and technology community 
needs to act now and meet the challenge 
of recruitment. However, it should not 
underestimate the effort it will take to 
employ efficient and effective recruitment 
strategies while achieving employment 
equity objectives. Science-based 
departments and agencies will have to 
obtain the necessary tools and resources 
from the government, including adequate 
forms of rewards, recognition and 
incentives. 


High-calibre staff need to be retained 


9.44 According to studies, testimony 
and documents such as the Report of the 
Senate Standing Committee on National 
Finance (February 1999), the 
government’s future capacity in science 
and technology depends on its ability to 
recruit and retain high-calibre scientists 
and technologists. Salary freezes, 
expenditure and work force reductions and 
their effects (increased workloads, lack of 
job security and reduced opportunities for 
promotion) and an expanding North 
American economy are leading some 
scientists and technologists to leave the 
public service or to question their 
commitment and loyalty to their 
employer. 


9.45 In this environment, the 
challenge for the government and the 
science-based departments and agencies is 
to offer their employees not only adequate 
compensation but also a stimulating work 
environment with varied, enriching 
assignments that enable them to contribute 
to society. 


9.46 The community must also rise to 
the dual challenge of recruiting promising 
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Exhibit 9.3 


Developing a Competency 
Profile for Science and 
Technology Managers - 
National Water Research 
Institute’s Approach 


young scientists and technologists and 
retaining highly qualified employees. 
Central agencies and science-based 
departments need to take immediate 
action. Without high-calibre employees in 
place to nurture and develop young 
scientists and technologists, there is a risk 
that the community will not achieve the 
results it expects from recruitment. 


Management Capability: Some 
Progress 


9.47 In our 1994 Report, we noted the 
need to develop a stronger and more 
effective management capability in 
research and development. We expected 
that this management capability would be 
developed strategically by identifying 
competency profiles that describe the 
skills and knowledge expected at various 
levels of science and technology 
management, from project leader to 
research establishment director, for 
example. We pointed out that these 
competency profiles could also be used to 
select supervisors, managers and members 
of the executive group, and as a reference 
point for assessing their performance. 


9.48 In its Science and Technology 
Blueprint for Human Resources 
Management, the Treasury Board 
Secretariat suggested developing a 
competency profile. It viewed this as an 
indispensable step in the integration of 
recruitment and training activities and the 


reward, promotion and compensation 
systems. In 1998, a working group looking 
into management and scientific 
development and training prepared a 
competency profile for science and 
technology managers. 


9.49 In addition, as a pilot project, 
Environment Canada’s National Water 
Research Institute recently developed a 
competency profile for its research and 
development (R&D) managers. Exhibit 9.3 
depicts the Institute’s approach. 


9.50 At the same time, most 
science-based departments developed their 
own competency profiles for general and 
R&D management positions that reflect 
their own organizations. During our 
consultations, we noted that most 
science-based departments used their own 
competency profiles instead of the one 
developed by the working group. 


9.51 At the end of our follow-up work, 
those profiles were being used solely to 
identify courses needed for training and 
development of science managers in 
science-based departments. We also noted 
that the Canadian Centre for Management 
Development had dropped the common 
training courses formerly offered to 
science and technology managers. 


9.52 In December 1998, the Science 
and Technology Senior Steering 
Committee on Human Resources 
approved the recommendations of the 
working group on management and 


develop and apply the competency profile. 


e Compilation of individual results. 


e Establishment of a discussion panel made up of senior managers, internal and external human 
resource experts and approximately 15 research and development managers who worked on 
various projects and whose performance was rated superior. 


e Discussion panel meetings that made it possible to put together a framework used to design, 


e Discussion panel meetings that made it possible to identify an array of success criteria 
experienced and conveyed by the research and development managers. 


e Confidential individual assessment of all the Institute’s managers against those success criteria. 


e Establishment and distribution of a dictionary containing a profile of nine competencies, 
including definitions, performance standards and suggested training and development methods. 


Source : Adapted from information provided by the Institute, 1997-1998. 
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scientific development that the 
community develop, in consultation with 
the Canadian Centre for Management 
Development, a training module for 
science and technology managers and 
identify in this module the training 
elements that can be offered by 
science-based departments. 


953 If the community is to increase 
the mobility of its science managers in the 
federal government, as it recommended to 
the Senior Steering Committee, in our 
view it needs to ensure that some of the 
management skills required are common, 
transferable and known to its members. 
For example, when the science and 
technology community defines the 
competencies of the manager of the 
future, it needs to distinguish between 
general and R&D management; and it 
needs to communicate those competencies 
to employees with or without science and 
technology backgrounds who aspire to 
management positions. 


9.54 According to the Blueprint, the 
science-based departments and agencies 
are responsible for co-ordinating the 
training of their science and technology 
managers. However, it does not specify 
the performance expectations in terms of 
science management capability; roles and 
responsibilities for monitoring 
performance; or corrective measures when 
performance falls short of expectations. 
As a first step, the competency profile 
proposed for the science and technology 
manager marks a shift toward the 
integrated approach suggested in the 
Blueprint. However, if the science and 
technology community is to act on its own 
recommendations, it will have to integrate 
and co-ordinate its activities related to 
developing management capability in 
research and development. 
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OSS In our opinion, current practices 
suggest a lack of consensus within the 
community on the management 
competency profile developed by the 
working group. This could eventually 
prevent the integration of recruitment and 
training activities and the reward, 
promotion and compensation systems 
envisioned in the Blueprint. 


Conclusion 


9.56 Overall, we think that the 
Treasury Board Secretariat, science-based 
departments and agencies and the science 
and technology community have shown 
leadership in the management of their 
human resources. Based on the strength of 
the community’s achievements and 
progress despite the difficult environment, 
some senior officials have suggested that 
the science and technology community 
could serve as a model across the federal 
government. 


POY | We believe that the work 
accomplished to date completes the 
planning phase. At the end of their work, 
the working groups set up to examine the 
key issues recommended to the Science 
and Technology Senior Steering 
Committee on Human Resources many 
ways to take action. However, there are 
still major challenges to be met. In some 
cases, changes in demographic profiles 
have worsened since 1994 and have 
accentuated the problems of rejuvenation 
and renewal. The science and technology 
community now needs to implement 
efficient and effective strategies to recruit 
and retain high-calibre staff and obtain 
from the government the tools and 
resources it needs to do so. 


If the community is to 
increase the mobility 
of its managers, it 
needs to ensure that 
some of the 
management skills 
required are common, 
transferable and 
known to its members. 


Some senior officials 
have suggested that 
the community could 
serve as a model 
across the federal 
government. 
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y About the Follow-up 


The Framework for the Human Resources Management of the Federal Science and Technology Community, 
released in 1996 by the Treasury Board Secretariat, was designed to serve as a collection of policies and tools 
that science managers could use to align their organizations and their scientific and technological personnel 
with the science direction and activities of their departments. 


Objective and Scope 


Our follow-up objective was to assess the extent of progress made by the science and technology community 
on two major issues we had raised in our 1994 Report (Chapter 11) and that were included in the Framework: 
renewal, rejuvenation and recruitment of scientific personnel, and improvement in management capability. 


We also assessed the progress made on issues raised in our 1996 Report Chapter 15, Federal Science and 
Technology Activities: Follow-up. Those issues included the need for a results-oriented, time-phased action 
plan and for effective mechanisms to share “best practices” and discuss issues of common interest. 


Approach 


We worked from a global perspective and we dealt mostly with the Science and Technology Unit at the 
Treasury Board Secretariat. However, we consulted stakeholders in science-based departments and agencies. 
We reviewed various reports issued by central agencies, science-based departments and the community. We 
did not audit the management of scientific personnel in science-based departments and agencies. 


To quantify certain results, we used the Treasury Board Secretariat’s information systems for the years 
1994-1995 to 1997-1998. More specifically, we analyzed work force changes in scientific and technological 
professional groups in the six main science-based departments for whom Treasury Board is the Employer. 
This approach was used so we could compare the results of our analyses with those carried out by the 
community and published in its report to the Science and Technology Senior Steering Committee on Human 
Resources that dealt with recruitment needs for 1998-2002. 


Audit Team 


Assistant Auditor General: Maria Barrados 
Responsible Auditor: Denise Coudry-Batalla 


For information, please contact Maria Barrados. 
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Appendix 


Management Structure Adopted by the Science and Technology Community 


A Response to Address Concerns Raised by the Auditor General in 1994 


A management structure has been put into place to implement the Framework for the Human Resources Management 
of the Federal Science and Technology Community. The structure was modified somewhat since 1996 when the La 
Reléve initiative was launched. La Reléve is an initiative aimed at modernizing the public service through the creation 
of a “workplace where people are valued, recognized, given opportunities for self-development, and treated in 
accordance with the core values of the public service”. 


| Clerk of the Privy Council and Head of the Public Service ; 
| Committee of Senior Officials 

| Science and Technology Community Sub-Committee 

| Science and Technology Senior Ree: Committee on Human Resources 


Working Group on Working Group on 
Classification and Recruitment and 
Salary Compression Rejuvenation 


Science and 
Technology Unit | 


Working Group on 
Rewards, Recognition 
and Incentives 


Working Group on | 
Management and 

Scientific Development 

and Training 


Working Group on 
Workforce and Mobility 


Science Managers, Scientists, Technologists and Union Representatives 


Clerk of the Privy Council and Head of the Public Service. The Clerk of the Privy Council has been formally 
recognized as the Head of the Public Service since 1993. By statute, the Head of the Public Service must report 
annually to the Prime Minister on the state of the public service. The Clerk chairs the Board of Governors of the 
Canadian Centre for Management Development and the Committee of Senior Officials. 


Committee of Senior Officials (COSO). The Committee, composed of deputy ministers of departments and deputy 
heads of agencies, provides advice and counsel to the Clerk of the Privy Council on a number of issues — including 
government-wide human resource issues. The Secretary of the Treasury Board and the President of the Public Service 
Commission are members of the committee. The Committee of Senior Officials has had significant involvement in 
initiatives aimed at renewing or modernizing the public service. In the context of La Reléve, sub-committees of COSO 
were created to champion, support or integrate departmental initiatives. A sub-committee composed of deputy 
ministers and chaired by a deputy minister from a science-based department was created to support initiatives by the 
science and technology community to address human resource issues and challenges faced by science-based 
departments and agencies on the eve of the 21st century. 


Report of the Auditor General of Canada — April 1999 9-17 


Management of Science and Technology Personnel: Follow-up 


Science and Technology Senior Steering Committee on Human Resources. The Senior Steering Committee is 
composed of assistant deputy ministers from science-based departments; representatives from organizations such as the 
National Research Council, the Treasury Board Secretariat and the Public Service Commission; and the Professional 
Institute of the Public Service of Canada, the bargaining agent for scientists. The Steering Committee is co-chaired by 
the Treasury Board Secretariat and an assistant deputy minister from a science-based department. The committee leads 
the implementation of the Framework and acts as “champion” for the various projects. The Science and Technology 
Unit at the Treasury Board Secretariat provides support to the committee. 


Treasury Board Secretariat. In addition to participating in the activities of the various projects teams, the Treasury 
Board Secretariat initially designated the senior officials responsible for the various projects. The Secretariat also 
designated the policy centres responsible for working and providing specialist support to project teams examining 
human resource issues and developing action plans. 


Working Groups. Initially, five working groups were established to examine human resource issues: management 
and scientific development and training; rewards, recognition and incentives; workforce and mobility; classification 
and salary compression; and recruitment and rejuvenation. The working groups comprise senior scientists, managers, 
engineers and technologists, as well as representatives of the major science-based departments, along with 
representatives of the Professional Institute of the Public Service of Canada, the Treasury Board Secretariat, the Public 
Service Commission and human resource specialists providing support. Generally, the working groups are led by 
departmental officials and are self-managed but with predetermined project plans and timeframes. The working groups 
are responsible for recommending solutions and for managing change and implementing subsequent activities such as 
pilot projects. 


Science and Technology Unit. Housed in Treasury Board Secretariat, the Science and Technology Unit is composed 
of two Treasury Board Secretariat employees and one departmental scientist, who represents the Professional Institute 
of the Public Service of Canada. The Unit supports project teams to reduce duplication and overlap while maintaining 
a process for ensuring that projects keep rolling. The Unit is also responsible for informing senior officials about the 
status of projects and establishing a communications strategy to keep departments and the science and technology 
community informed about the implementation of the Framework. 


Source: Adapted from information provided by the Treasury Board Secretariat. 
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Chapter 10 


Indian and Northern Affairs 
Canada 

Funding Arrangements for First 
Nations: Follow-up 


The follow—up work was conducted in accordance with the legislative mandate, 
policies and practices of the Office of the Auditor General. These policies and 
practices embrace the standards recommended by the Public Sector Accounting 
and Auditing Board (PSAAB) of the Canadian Institute of Chartered Accountants. 
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pay 


Indian and Northern Affairs 
Canada 

Funding Arrangements for First 
Nations: Follow-up 


Main Points 


10.1 Indian and Northern Affairs Canada still needs to better match the level of flexibility attached to funding 
arrangements with the willingness and ability of First Nations to assume responsibility for billions of dollars they 
receive annually through the Department. 


10.2 The Department is not taking adequate steps to ensure that allegations of wrongdoing, including 
complaints and disputes related to funding arrangements, are appropriately resolved. Redress (resolution 
mechanisms) needs to be improved as an element of accountability. 


10.3 The Department has stated that it has slowed the rate at which the Financial Transfer Arrangement (FTA), 
anew type of funding arrangement, is being implemented in order to address issues concerning the willingness 
and ability of First Nations to adopt it. The Department still has a long way to go if it wishes to achieve its 
objective of implementing the FTA as the appropriate funding mechanism to replace other types of funding 
arrangements. It will need to find ways to expedite the conversion process while improving co-ordination of 
funding with other federal departments. 


Background and other observations 


10.4 Funding arrangements are a key element in the relationship between First Nations and the federal 
government. Parliament appropriates about $4 billion annually to Indian and Northern Affairs Canada to fund 
several programs for First Nations communities. The programs include social assistance, education, capital 
facilities, housing, and economic development for approximately 600 First Nations and other Aboriginal groups. 


10.5 In our November 1996 Report, we made recommendations relating to funding arrangements, including 
the FTA, and to the co-ordination of funding across federal departments. We also made recommendations on 
accountability issues and on the suitability of funding arrangements for the needs of the Department and First 
Nations. 


10.6 This follow-up focusses on the Department’s implementation of our 1996 recommendations. 


The Department believes that with respect to redress, the follow-up extends beyond the issues raised in 1996; 
however, it has acknowledged that continuing effort is needed to improve certain aspects of funding 
arrangements, including accountability. 
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Introduction 


The significance of funding 
arrangements 


10.7 Parliament appropriates about 

$4 billion annually to Indian and Northern 
Affairs Canada to fund several programs 
for First Nations communities. The 
programs include social assistance, 
education, capital facilities, housing, and 
economic development for approximately 
600 First Nations and a number of other 
Aboriginal groups. 


10.8 Funding arrangements for First 
Nations are mainly in the form of 
contribution agreements. These 
agreements set out specified terms and 
conditions for expenditures, including 
community service standards and 
accountability and performance 
expectations. 


10.9 The nature of funding 
arrangements between First Nations and 
the Government of Canada reflects the 
evolving relationship between the federal 
government and Aboriginal peoples by 
providing mechanisms to address the 
willingness and the capacity of First 
Nations to manage their affairs. These 
arrangements set out the means by which 
First Nations and the federal government 
ensure that programs are delivered, as 
agreed between parties, to the benefit of 
those mainly living on reserves. As of 
1996, six types of funding arrangements 
were being used in the Department, each 
reflecting different responsibilities and 
expectations between parties. 


The 1996 audit issues 


10.10 In 1996 we carried out an audit of 
funding arrangements and made 
recommendations relating to: 


e anewly developing type of funding 
arrangement — the Financial Transfer 
Arrangement (FTA); 
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e the co-ordination of funding across 
federal departments; 


e the suitability of funding 
arrangements for meeting departmental 
and recipient needs, including the 
willingness and capacity of recipients to 
enter into these agreements; and 


e accountability issues and other 
matters. 


Departmental commitment to 
improvements 


10.11 In December 1996, shortly after 
publication of our audit report, the 
Standing Committee on Public Accounts 
met to deliberate the issues. At the 
hearing, the Department indicated that it 
had taken measures during the audit and 
would continue to take further steps to 
address the audit recommendations. 


10.12 Inresponse to the 1996 Report of 
the Royal Commission on Aboriginal 
Peoples, the Department indicated its 
commitment in Gathering Strength — 
Canada’s Aboriginal Action Plan, to 
develop stable financial arrangements that 
promote accountability and foster 
self-reliance of Aboriginal communities. 


10.13 In its 1998-99 Report on Plans 
and Priorities, the Department states that 
developing a new fiscal relationship is a 
priority and that the ultimate outcome of 
efforts to achieve this and strengthen 
accountability will be the establishment of 
government-to-government fiscal transfers 
that provide more stable funding, integrate 
self-generated revenues and taxation with 
strong information systems, and enhance 
local accountability. 


Focus of the follow-up 


10.14 ~The overall objective of the 
follow-up was to determine progress in 
implementing the recommendations made 
in our 1996 Report Chapter 33, Funding 
Arrangements for First Nations. We took 
into account the Department’s 
commitments and the actions taken in 
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Canada’s Aboriginal 
Action Plan 
emphasizes the need 
to promote 
accountability and 
foster self-reliance 
through funding 
arrangements. 
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response to our 1996 audit, including the 
progress the Department reported up to 
December 1998. Further details on the 
follow-up are found at the end of the 
chapter in About the Follow-up. 


Observations 


Initiatives Undertaken Since 1996 


10.15 Several initiatives are being 
implemented. Indian and Northern Affairs 
Canada embarked on a new Financial 
Transfer Arrangement (FTA) as the 
replacement, by March 1998, of most 
other funding arrangements. In its 
published response to our 1996 audit, the 
Department pointed to the development of 
the new FTA as “an important step in the 
right direction” in addressing the audit 
issues. 


10.16 The FTA is intended to: 


e strengthen accountability; 


e enable First Nations to better meet 
community needs; 


e establish a stable funding base; and 
e achieve better value for money. 


10.17 Steps by the Department toward 
improving funding arrangements include: 


e the development of a First Nations 
accountability and management 
self-assessment workbook and a 
requirement that every First Nation 
self-assess and report its accountability 
and management regimes to the 
Department as a prerequisite to receiving 
departmental funding; 


e the use of development plans to 
improve accountability and management 
where indicated in the applicable First 
Nations self-assessments; 


e support of external initiatives to 
enhance financial capacity and 
governance in First Nations communities; 


e the development of national 
allegations (redress) guidelines; and 


e the development of a generic 
interdepartmental funding agreement to 
better co-ordinate First Nations funding 
across federal departments. 


10.18 Since all of the initiatives are 
relatively new, tests of their effectiveness 
during the follow-up were limited. The 
observations that follow do not necessarily 
apply equally to each of the four 
departmental regions that we visited. 


The New Financial Transfer 
Arrangement 


10.19 In 1996, the Department set a 
target of March 1998 for finalizing the 
conversion of all funding arrangements to 
Financial Transfer Arrangements. An FTA 
is typically a five-year funding contract 
that is more flexible than the other 
arrangements presently in use. However, 
along with flexibility and greater certainty 
of funding levels over a longer term, First 
Nations take on greater risks. Surplus 
funds in any one year may be carried over 
to the next year but shortfalls must be 
covered by First Nations. 


10.20 Progress in adopting the FTA is 
considerably behind the Department’s 
target. According to the Department, 117 
First Nations and tribal councils, less than 
20 percent, were using FTAs as of January 
1999. At the rate of progress since 1996, it 
will take at least another 10 years to 
completely replace existing funding 
arrangements. For example, the 
Department has identified 39 recipients 
whose current or expiring Alternative 
Funding Arrangements are being 
considered for extension beyond the 
normal expiry date because of issues to be 
addressed relating to capacity or 
willingness to adopt the FTA. 


10.21. The Department has stated that it 
has slowed the rate at which the FTA is 
being implemented to address issues 
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concerning the willingness and ability of 
First Nations to adopt it. 


10.22 Information obtained during the 
follow-up suggests four major challenges 
to progress in implementing the FTA. 
These are consistent with the audit issues 
reported in 1996. 


e There is a perception among many 
First Nations that they will receive less 
overall funding under the FTA than 
through existing funding arrangements. 


e Some First Nations believe that they 
did not have sufficient input to the design 
of the FTA. 


e Some First Nations are unable or 
unwilling to assume the additional 
responsibilities (and control) that may 
accompany the more flexible FTA. 


e There is a lack of training 
opportunities to enable First Nations to 
deal with the requirements of the FTA. 


10.23. We further noted that the 
Department does not have a plan and 
strategy to facilitate progress in adopting 
FTAs, where appropriate. We could find 
no plans in the four departmental regions 
we visited that would address this, or that 
would apply to any other new funding 
arrangement that may be considered. 


10.24 =Sucha plan would include: 


e the considerations and approach to 
be used in achieving the objective; 


e the identification of departmental 
and First Nations risks; 


e the development of a realistic 
conversion timetable that all parties could 
agree to; and 


e the identification of alternative 
courses of action (including the possible 
replacement of the FTA if circumstances 
so dictate) to address challenges along the 
Way. 
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10.25 Finally, we believe that the FTA, 
or any other funding arrangement that 
may evolve in the future, will not achieve 
the necessary objectives if there is no will 
and effort by First Nations and the 
Department to work together effectively 
toward the right arrangement for the 
circumstances. 


Co-ordination of Federal Funding 
to First Nations 


10.26 The Department is attempting to 
better co-ordinate funding to First Nations 
with other federal departments. Most, if 
not all, First Nations appear to be eligible 
for funding from several federal 
departments. Appropriate co-ordination 
can benefit all the parties in several ways, 
including increased efficiency of 
administration, appropriate accountability 
and reduced duplication of reporting 
requirements. 


10.27. With a view to improving 
co-ordination, the Department developed 
another funding arrangement, the 
Canada/First Nations Funding 
Arrangement (CFNFA). The CFNFA is the 
current generic arrangement for 
multi-departmental funding as represented 
by the national model introduced in 
December 1998. The Department believes 
that the CFNFA will replace and improve 
upon the FTA, which is already being used 
for multi-departmental funding. 


10.28 The Department’s experience 
with the development of funding 
arrangements since 1986 shows that 
progress has been slow. Officials in one 
regional office believed that the proposed 
CFNFA was more restrictive than the FTA 
and that First Nations were not provided 
adequate opportunity for input to its 
design; we reported on this issue in 1996 
regarding the FTA. It is too early to 
conclude whether the CFNFA approach 
will be effective. 
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Indian and Northern 
Affairs Canada is 
missing many of the 
self-assessments it 
needs to tailor funding 
arrangements. 


First Nations Self-Assessments of 
Accountability and Management 


10.29 = In 1996, we reported that almost 
one third of First Nations and tribal 
councils were in financial difficulty. The 
Department has recognized the need to 
respond to the capacity and willingness of 
First Nations to assume the challenges of 
greater responsibility and accountability. 
In this regard, funding arrangements must 
be tailored to meet the needs of both 
parties. 


10.30 The Department developed a 
40-page self-assessment workbook to be 
completed by First Nations. A wide range 
of topics are covered, including 
community leadership and structures, 
governance, human resources 
management, program management and 
redress. 


10.31 A major purpose of the 
self-assessment is to provide information 
necessary to assure Parliament and First 
Nations that an adequate system of 
accountability for the delivery of funded 
programs 1s in place. Where a 
self-assessment identifies areas for 
improvement, a development plan is 
required to address specific deficiencies. 


10.32 When properly completed and 
applied, the self-assessments and 
development plans can also play an 
important role in managing risks and 
enhancing First Nations capacity to 
assume greater responsibilities. The 
implementation of the development plan 
becomes a requirement under the funding 
arrangement. Ultimately, these 
self-assessments should advance the 
financial relationship between the parties 
toward increasing self-governance 
responsibilities. 
10.33 We therefore expected that: 

e the Department would determine the 
reasonableness of completed 
self-assessments; 
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e First Nations would properly approve 
and authorize their self-assessments; 


e adequate development plans would 
be prepared and applied where indicated; 
and 


e an appropriate database on 
self-assessments and development plans 
would be maintained to help measure 
progress against the objectives of the FTA 
and other funding arrangements. 


Self-assessment issues 


10.34 In 1996, the Department wrote to 
First Nations chiefs and councils 
requesting that they complete an 
assessment of their accountability and 
management regimes. In our follow-up, 
we selected a sample of self-assessments 
related to the FTA and other funding 
arrangements. We found inconsistencies in 
how the Department had determined the 
reasonableness of the information 
provided. For example, there was often no 
evidence that the Department had assured 
itself that positive responses (those that 
indicated no deficiencies) were justified. 
And positive responses were received 
more often than negative ones. While it 
should not be necessary for the 
Department to document a detailed review 
in every self-assessment, some level of 
corroboration is needed on a test basis and 
for high-risk cases. 


10.35 We noted that none of the 
self-assessments that we reviewed were 
signed by the First Nations, nor was there 
evidence in the Department of Band 
Council Resolutions that could attest to 
the authority and commitment of the 
self-assessments provided. In our view, the 
self-assessments have limited value unless 
there is evidence that they are accepted 
and endorsed by First Nations leadership. 


10.36 We further noted that many 
self-assessments were missing. For 
example, one departmental region had 
received completed self-assessments from 
only 14 of 61 First Nations (23 percent). 
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10.37 ‘In another region, about 50 of 
130 First Nations (38 percent) had 


completed self-assessments. In this region, 


none of the four FTAs with First Nations 
were supported by self-assessments. 
Instead, the regional office continued to 
rely on older assessments that were 
completed many years earlier and that 
related to a different type of funding 
arrangement. In our view, this reliance is 
inappropriate given the Department’s 
objectives for improved accountability 
and enhanced capacity building through 
the FTA. In the absence of departmental 
data, the Department is unable to 
determine whether it is relying on 
potentially obsolete information for other 
funding arrangements in this region and in 
other regions. 


10.38 Further, officials in the 
above-noted region observed that the 
requirement for self-assessments was 
poorly managed. They noted that the 
departmental directive and First Nations 
guidelines were issued two years after 
First Nations were formally notified of the 
requirement to provide self-assessments. 
Furthermore, in 1998, they were 
considering the establishment of a project 
team to deal exclusively with issues such 
as completion and evaluation of 
self-assessment reports and negotiation of 
management development plans for 
inclusion in funding arrangements. 


10.39 The Department reported that, 
nationally, 407 of the required 644 
self-assessments (63 percent) had been 
completed by October 1998. Although the 
Department maintains databases on 
various activities related to funding 
arrangements, there is no national 
database that discloses essential 
information from the self-assessments 
received. This should include information 
by type of funding arrangement, by year 
in which the self-assessments were 
prepared, and showing whether a 
development plan was necessary and 
completed. 
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10.40 Consequently, the Department is 
unable to determine overall progress in 
tailoring funding arrangements to the 
needs of all parties, an issue noted in our 
1996 audit. 


Development plans 


10.41 Completed self-assessments are 
used to prepare development plans. 
However, it was often questionable 
whether the plans were sufficiently 
comprehensive to effectively address the 
identified deficiencies. For example, in 
many cases the plan was simply a listing 
of the negative responses to the 
assessment questionnaire. 


Development plans 
require greater rigour 
to address First 
Nations needs. 


10.42 We believe that development 
plans should explicitly identify: — 


e the nature and extent of the 
necessary remedial action; 


e the expertise needed to achieve this 
action; 
e the approach to be taken; and 


e the time frames for completion, and 
other related matters. 


10.43 Finally, we believe that the 
Department and First Nations will need to 
re-examine their experience with 
self-assessments at appropriate intervals to 
determine the ongoing effectiveness of the 
assessments, the reasonableness of the 
expectations and their compatibility with 
the needs of First Nations. 


Redress and Accountability 


10.44 = In 1996, the Department 
published certain principles of 
accountability, including redress. In its 
guidance to First Nations on how to 
complete the accountability and 
management self-assessments, the 
Department elaborates on redress as a key 
principle of accountability as follows: 


Formalized review and appeal rules 
and procedures are in existence for 
operating programs and key 
governance functions that impact on 
the rights/entitlements of individuals 
and/or the community. 
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Funding arrangements 
contain provisions for 
accountability, 
including redress. 


Effective redress 
includes the adequate 
resolution of 
allegations. 


This means that First Nations and their 
members will have the opportunity for an 
objective review and resolution of 
allegations. 


10.45 The concept is that strong redress 
mechanisms in First Nations communities 
will enhance accountability of all parties. 
The terms and conditions in funding 
arrangements include redress as a 
principle of accountability to be followed. 


10.46 The Department considers that its 
responsibility to manage allegations stems 
from its role as a major provider of 
funding to First Nations, and from its 
obligations under the /ndian Act. It also 
believes that under its legislative mandate, 
there are limits to the actions it may take. 


10.47 Effective redress includes the 
adequate resolution of allegations. This 
means that the Department would take 
proper action, including, where 
appropriate, referral of the allegations to 
other parties. The Department recognizes 
that the ultimate responsibility for 
addressing an allegation may lie within a 
First Nation, the Department, another 
funding agency, or a law enforcement or 
other investigative authority. 


10.48 A major and challenging 
departmental decision applicable to every 
allegation the Department receives is to 
determine the extent of its responsibility 
for resolution and where the case should 
be referred, internally and possibly 
externally. 


Significance of allegations 


10.49 The Department reported that it 
had received over 300 allegations relating 
to 108 First Nations during the two-year 
period prior to this follow-up. The 
allegations relate to such matters as social 
assistance issues, mismanagement of 
funds and other concerns. Because 
departmental data are incomplete, the 
total reported is the minimum known 
number of allegations. 


10.50 Allegations can be challenging to 
resolve. They can be reported for frivolous 
or politically motivated reasons, or they 
can be legitimate and have serious 
consequences for an entire departmental 
program, for First Nations members and 
for living conditions on reserves. They 
may reflect criminal activity or simply 
managerial negligence without intent to 
deceive. Serious allegations that are 
proved could impair the relationship 
between First Nations and the 
government. 


10.51 Different types of allegations 
may require different action. For example, 
reported issues of non-compliance with a 
funding arrangement or regulation and 
alleged deficiencies in program 
management may require different 
approaches for disposition. As well, some 
allegations are best resolved by the First 
Nation community. Inappropriate 
disposition of allegations can be 
detrimental to effective management of 
funding arrangements and can 
unnecessarily overburden other agencies. 


10.52 We performed a selective 
overview of how the Department manages 
its caseload of allegations. As part of the 
overview, we examined a sample of 
allegation files in the Department; 
however, we did not investigate any 
allegations, and we do not take a position 
on any of them. Nor did we review redress 
practices in First Nations. These are 
matters for the Department, other 
authorities and First Nations to act upon. 


Guidelines for managing allegations 


10.53 In February 1998, the 
Department issued a national guideline to 
senior officials on how to deal with 
allegations. The guideline defines and 
categorizes the types of allegations, 
outlines the procedures to be followed, 
and discusses the importance of 
departmental follow-up and related 
matters. 
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10.54 There are also draft directives on . 
allegations that have been developed by 
some departmental regional offices. 


the conditions leading to the 
allegations; 


e the areas of high and low risk within 
10.55 The way in which an allegation is | and among different First Nations; and 
processed and disposed of will always 
require careful judgment, which can vary 
even in similar cases and among different 
departmental officials. We therefore 
expected to find departmental guidelines 
on: 


e the impact on accountability and 
funding arrangements. 


10.59 In response to our request for 
information on allegations, one regional 
office reported that it did not know how 
many allegations it had received during 
the past two years. 


Little guidance has 
been provided in the 
Department on how to 


e how to evaluate the merit of an 
allegation; 


10.60 Another regional office maintains 


e how to decide what additional 
information is needed; 


¢« who should collect any additional 
information; and 


e how to decide whether the allegation 
should be resolved by the Department or 
referred to other authorities or First 
Nations. 


10.56 Although we found that guidance 
is given on how to control the flow of 
documents and communications, there is 
little guidance on how to evaluate an 
allegation. 


10.57 We believe that departmental 
guidelines need to further clarify the 
responsibilities of the Department and 
First Nations respecting the determination 
of merits of an allegation, the nature of 
the risks, the setting of priorities, the 
identification of alternative actions, and 
preventative measures. 


Information for managing allegations 


10.58 Although some departmental 
regions maintain a control listing of cases 
in progress, the Department does not have 
a national reporting system to help 
manage allegations. Consequently, it does 
not have an overall picture of the nature 
and frequency of allegations that shows: 


e their ultimate disposition; 


a listing of allegations but there was no 
summary report available that disclosed 
the initial inventory of allegations, the 
quantity of monthly intake and 
dispositions, the category of the 
allegations and programs affected, their 
ultimate disposition, and the current 
balance of unresolved allegations and 
their age. We believe this information is 
necessary for managing allegations. 


10.61 We further believe that the 
absence of a national reporting system for 
allegations and of a self-assessments 
database makes it more difficult for the 
Department to obtain the assurance it 
needs on the validity of the 
self-assessments and the suitability of the 
funding arrangements. It is also more 
difficult to identify where and how 
redress, and hence accountability, can be 
enhanced. 


10.62 With respect to the coverage of 
redress by the self-assessment workbook 
(see paragraph 10.30), the self-assessment 
questionnaire contains only one question. 
This question asks whether a dispute 
resolution and appeal process to address 
complaints is in place. In our view, this is 
a cursory and inadequate attempt to 
identify and evaluate redress. We believe 
that additional information on the 
structure of the redress system, how it is 
applied, and the safeguards in place needs 
to be included in the self-assessment 
questionnaire. 


evaluate an allegation. 
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Allegation cases 


10.63 We examined a sample of 
allegations to determine whether case files 
were active. We did not evaluate the 
appropriateness of the action or the 
disposition of the cases. 


10.64 We found that there was activity 
in most cases in this sample. However, in 
a region with a large number of cases, it 
was often difficult to determine what 
exactly was being done and what was 
being achieved. This was because of 
ambiguous file notations that contained 
unclear references to inquiries and other 
actions. Because of deficient 
documentation and the absence of 
documentation standards, the Department 
is at risk should it need to demonstrate to 
other parties that it acted appropriately. 


10.65 We believe that the Department 
needs to develop and apply an appropriate 
standard of documentation for use by all 
regions. 


First Nations Perspectives on 
Funding Arrangements 


10.66 In 1996, we surveyed and 
reported the opinions of a sample of First 
Nations on whether funding arrangements 
with the Department needed to be 
improved and, if so, how this could be 
achieved. In this follow-up, we again 
surveyed a sample of First Nations, 
including those that had responded to the 
1996 survey. For the most part, both 
surveys contained identical questions. 


10.67 Thirty-eight First Nations 
responded to this survey (forty in 1996). 
In both surveys, we received responses 
from several regions and from 
different-sized communities, and they 
related to different types of funding 
arrangements. 


10.68 No audit of the survey responses 
was performed and, since the opinions 
provided by the respondents do not 
necessarily reflect the opinions of those 
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who did not respond or who were not 
surveyed, no generalizations can be made. 
However, we believe that the views 
presented below provide valuable 
perspectives that need to be considered in 
the evolving financial relationship between 
the government and First Nations. 


10.69 Of the 38 First Nations who 
responded, 25 believe that their current 
funding arrangements (FTA and others) 
meet their needs for flexibility in 
allocating funds and in implementation; 
33 stated that the terms and conditions of 
their funding arrangements were clear and 
reasonable. 


10.70 However, 15 First Nations 
believed that they did not have sufficient 
input to the terms and conditions; 14 
stated that improvement was needed in the 
co-ordination of funding across federal 
departments; 11 believed that the 
Department’s reporting requirements were 
overly burdensome. 


10.71 Other comments include the 
following: 


e “There was really no other choice than 
the FTA, which we felt was forced on us.” 


e “It would be important for Indian 
and Northern Affairs Canada to involve 
First Nations communities in the design of 
new funding arrangements that would 
allow for the proper reporting of results. 
This would help remove current fears and 
negative perceptions that First Nations 
have about the Department’s unilateral 
approach on this issue. The FTA 
agreement has its benefits and limitations 
and it is important for First Nations to be 
well informed and prepared before 
considering this funding agreement.” 


e “Need more clearly defined reporting 
requirements...” 


e “The clauses on accountability in the 
funding arrangement are reasonable but 
implementation by First Nations and 
verification by the federal government is 
sometimes weak.” 


10.72 Overall, the perspectives are not 
substantially different from those we 
reported in 1996. The variety of views, 
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both positive and negative, continues to 
imply that funding arrangements need to 
be tailored to meet the different needs of 
First Nations as well as those of the 
Department. We would expect this need to 
be addressed as the Department continues to 
implement its initiatives for improvement. 


Conclusion 


10.73 Indian and Northern Affairs 
Canada has taken action to address our 
1996 recommendations and is committed 
to ongoing improvements. More rigorous 
attention is still needed, however, in better 
matching the level of flexibility attached 
to funding arrangements with the 
willingness and ability of First Nations to 
assume responsibility for funds provided. 
In addition, the Department’s role with 
respect to redress needs to be clarified, 
and guidelines on the management and 
disposition of allegations further 
developed. 


10.74 The Department will also need to 
develop an appropriate database for First 
Nations Self-assessments and a national 
reporting system for allegations. 


10.75 We believe that if improvements 
to funding arrangements are not achieved, 
it will be more difficult for the 
Department to successfully manage the 
risks associated with them, such as the 
risk of failing to improve living conditions 
on reserves in a cost-effective way. As 
well, the ability of some First Nations to 
strengthen their capacity to become more 
self-reliant will be hindered. 


Department’s response: The Auditor 
General acknowledges the Department’s 
continuing efforts to address the 1996 
recommendations. In particular, the 
Department accepted that First Nations 
input needed to be considered in 
addressing the most appropriate type of 
funding arrangements. This is consistent 
with Gathering Strength — Canada’s 
Aboriginal Action Plan, which commits 
this department and other federal 
departments to work with Aboriginal 


peoples and governments to build a 
renewed partnership and a new fiscal 
relationship. Accordingly, the Department 
has adopted a plan to consult extensively 
with First Nations and other government 
departments over the next year on the 
Canada/First Nations Funding Agreement. 
This agreement will be introduced 
gradually, commensurate with First 
Nations acceptance and capacity to 
manage under this type of multi-year 
funding arrangement. As well, the 
Department will increase the level of 
effort required to complete the 
management and accountability 
assessments and attendant development 
plans. The Department is encouraged to 
note that of the 38 First Nations that 
responded to the survey, 25 (66 percent) 
felt that their current funding 
arrangements met their needs for 
flexibility in allocating funds and 33 

(S7 percent) felt that the terms and 
conditions of their funding arrangements 
were clear and reasonable. 


to improve funding 
arrangements to 
manage risks and 


of First Nations. 


As reported in the 1996 audit, the 
Department introduced the accountability 
principles of transparency, disclosure and 
redress. These have been accepted by First 
Nations and have been incorporated into 
all funding arrangements. In the context of 
the accountability framework required by 
First Nations, redress means an internal 
mechanism or process whereby citizens 
can appeal decisions affecting them. The 
Department's view ts that the Auditor 
General has broadened the scope of this 
follow-up audit (vis-a-vis redress) well 
beyond the context of the original 1996 
audit by investigating how the Department 
manages allegations, many of which are 
unrelated to the management of funding 
arrangements. Nevertheless, the 
Department is presently considering the 
need for a national reporting system and 
will give due consideration to the 
observations concerning allegations, 
which have been included under the 
section of this chapter entitled Redress 
and Accountability. 


Indian and Northern 
Affairs Canada needs 


enhance self-reliance 
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Mag 1 the Follow-up 


Objective 

The objective of this follow-up was to determine the progress made by Indian and Northern Affairs Canada in 
implementing the recommendations made in our 1996 Report Chapter 33, Funding Arrangements for First 
Nations. 

Scope 

The follow-up concentrated on: 

e the implementation of the new Financial Transfer Arrangement (FTA); 

e the co-ordination of funding to First Nations with other federal entities; 


e the completion of First Nations self-assessments of their management and accountability regimes and the 
use of such assessments by the Department; and 


e the development and application of certain accountability principles, such as redress. 


The follow-up was performed in four departmental regions that were responsible for planned contributions 
totalling about $2 billion to First Nations and other recipients during 1998-99. This represents about 
60 percent of the total planned contributions of all regions for the year. 


In addition to reviewing selected departmental initiatives, we examined 14 FTAs. At the time of our 
follow-up, these arrangements had an estimated value of more than $500 million over their terms, which in 
some cases was five years. 


We also reviewed 22 First Nations self-assessments of their accountability and management regimes, 
including those applicable to the FTAs and other funding arrangements. Further, we inquired into redress 
practices in all nine departmental regions and reviewed 25 allegations files selected from four regions. 


In addition, we obtained the written perspectives of a sample of First Nations on funding arrangements 
currently in use. 


Audit Team 


Assistant Auditor General: Don Young 
Principal: Grant Wilson 
Director: Ted Bonder 


Martha Lywak 
Suzanne Moorhead 


Marjorie Pound 


For information, please contact Grant Wilson. 
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